May 27 15:57:34.010043 np0000001523 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service.
May 27 15:57:35.516064 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.common.config [-] Logging enabled!
May 27 15:57:35.516468 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev277
May 27 15:57:35.516468 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=91387) setup_logging /opt/stack/neutron/neutron/common/config.py:125}}
May 27 15:57:35.517245 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting...
May 27 15:57:35.517462 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=91387) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:511}}
May 27 15:57:35.517533 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=91387) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:524}}
May 27 15:57:35.517594 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=91387) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:543}}
May 27 15:57:35.517808 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=91387) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:511}}
May 27 15:57:35.517808 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=91387) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:524}}
May 27 15:57:35.517808 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.manager [-] Loading core plugin: ml2
May 27 15:57:35.820923 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.820986 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.821057 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.821719 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.821792 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.821846 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.965639 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.965760 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.965851 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.975637 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.975637 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:35.975637 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.030989 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.031103 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.031230 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.039835 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.040128 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.040128 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.112008 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x73e77cb80fe0> agent after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.112102 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x73e77cb80fe0> agent after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.112171 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _add_segment_host_mapping_for_segment at 0x73e77cb81080> segment precommit_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.112243 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _delete_segments_for_network at 0x73e77cb81120> network precommit_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.112551 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _prevent_segment_delete_with_port_bound at 0x73e77cb82340> segment before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167122 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._before_network_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> network before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167232 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function DVRDbMixin._delete_mac_associated_with_agent at 0x73e77cb4df80> agent before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167232 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> network before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167297 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> port before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167361 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> port before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167469 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> segment precommit_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167512 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> segment precommit_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167568 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> segment after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167641 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> segment after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167738 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> network after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167793 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> network precommit_delete 0, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167852 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> port provisioning_complete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167908 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_create of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> rbac-policy before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.167964 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_delete of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> rbac-policy after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.168022 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._retry_binding_revived_agents of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> agent after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.168088 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> subnet after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.168142 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> subnet precommit_delete 0, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.168204 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> rbac-policy before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.168603 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> rbac-policy before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.168603 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> segment_host_mapping after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.168603 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin.delete_agent_notified of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> agent after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.168603 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> rbac-policy before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.168603 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> rbac-policy before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.168782 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73e77deb2390>> rbac-policy before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.169105 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=91387) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 27 15:57:36.169105 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=91387) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 27 15:57:36.169105 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=91387) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 27 15:57:36.169105 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=91387) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 27 15:57:36.169210 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=91387) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 27 15:57:36.169210 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=91387) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 27 15:57:36.169438 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve']
May 27 15:57:36.170445 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public']
May 27 15:57:36.171242 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
May 27 15:57:36.171882 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]})
May 27 15:57:36.171953 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan']
May 27 15:57:36.172115 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan'])
May 27 15:57:36.172205 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve']
May 27 15:57:36.172323 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos']
May 27 15:57:36.172872 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos']
May 27 15:57:36.172929 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos']
May 27 15:57:36.173023 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn']
May 27 15:57:36.213313 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn']
May 27 15:57:36.213417 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn']
May 27 15:57:36.213458 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
May 27 15:57:36.214161 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x73e77cbcade0>> floatingip before_response 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.214252 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x73e77cbcade0>> port before_response 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:36.243736 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
May 27 15:57:36.243843 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
May 27 15:57:36.243881 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve'
May 27 15:57:36.244028 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)]
May 27 15:57:36.294557 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO dbcounter [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Registered counter for database neutron
May 27 15:57:36.345167 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG oslo_db.sqlalchemy.engines [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91387) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}}
May 27 15:57:36.784318 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG dbcounter [-] [91387] Writer thread running {{(pid=91387) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
May 27 15:57:36.961474 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Initializing driver for type 'local'
May 27 15:57:36.961604 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Initializing driver for type 'vlan'
May 27 15:57:37.065992 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] VlanTypeDriver initialization complete
May 27 15:57:37.066110 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Initializing extension driver 'port_security'
May 27 15:57:37.066239 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.extensions.port_security [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] PortSecurityExtensionDriver initialization complete
May 27 15:57:37.066239 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Initializing extension driver 'qos'
May 27 15:57:37.066402 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.plugins.ml2.extensions.qos [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QosExtensionDriver initialization complete {{(pid=91387) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}}
May 27 15:57:37.066467 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Initializing mechanism driver 'ovn'
May 27 15:57:37.066527 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Starting OVNMechanismDriver
May 27 15:57:37.068007 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver.pre_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> process before_spawn 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.068139 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver.post_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> process after_init 55550000, True {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.068246 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._add_segment_host_mapping_for_segment of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> segment after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.068319 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver.create_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> segment after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.068440 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver.delete_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> segment after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.068484 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._validate_allowed_address_pairs of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> allowed_address_pair before_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.068600 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> security_group precommit_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.068700 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._update_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> security_group after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.068753 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> security_group after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.068859 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> security_group precommit_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.068933 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> security_group after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.069046 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._create_sg_rule_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> security_group_rule precommit_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.069171 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> security_group_rule after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.069279 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> security_group_rule before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.069406 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> address_group after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.069515 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> address_group after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.069649 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73e77c8b0e00>> address_group after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.069826 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method DriverBase._register of <neutron.services.qos.drivers.ovn.driver.OVNQosDriver object at 0x73e77c54d9d0>> qos_plugin after_init 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.069974 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNTrunkDriver.register of <neutron.services.trunk.drivers.ovn.trunk_driver.OVNTrunkDriver object at 0x73e77c824770>> trunk_plugin after_init 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.070145 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method DriverBase._register of <neutron.services.logapi.drivers.ovn.driver.OVNDriver object at 0x73e77b260a10>> logging-plugin after_init 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.070600 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.services.logapi.drivers.ovn.driver [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] OVN logging driver registered
May 27 15:57:37.070738 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <function register.<locals>._register_pvlan_driver at 0x73e77baf8c20> pvlan_plugin after_init 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.072004 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.plugins.ml2.plugin [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=91387) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 27 15:57:37.072259 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260c50>> port after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.072381 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260c50>> port after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.072478 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260c50>> port after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.072572 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260c20>> subnet after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.072673 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260c20>> subnet after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.072765 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260c20>> subnet after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.072949 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260b60>> network after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.072949 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260b60>> network after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073105 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260b60>> network after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073181 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260b90>> security_group after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073263 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260b90>> security_group after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073398 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260b90>> security_group after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073501 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260bc0>> security_group_rule after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073571 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260bc0>> security_group_rule after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073612 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260bc0>> security_group_rule after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073759 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260b00>> address_group after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073836 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260b00>> address_group after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073898 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73e77b260b00>> address_group after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.073934 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ML2 OVO RPC backend initialized. {{(pid=91387) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}}
May 27 15:57:37.074720 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.quota [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x73e77b2625d0>.
May 27 15:57:37.074818 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.plugin [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Modular L2 Plugin initialization complete
May 27 15:57:37.074935 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading services supported by the core plugin {{(pid=91387) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}}
May 27 15:57:37.075024 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.plugins.ml2.managers [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Got port-security extension from driver 'port_security'
May 27 15:57:37.075282 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=91387) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}}
May 27 15:57:37.075381 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading Plugin: ovn-router
May 27 15:57:37.084210 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._delete_router_remove_external_gateways of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73e77bf35160>> router before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.084317 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x73e77bf80900> port before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.084442 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x73e77bf80900> port after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.084553 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method L3_NAT_dbonly_mixin._precommit_delete_port_callback of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73e77bf35160>> port precommit_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.084605 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_internal_ip_change_for_fip at 0x73e77c9ae340> port before_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.084716 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback at 0x73e77c9ae2a0> port before_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.084800 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method RouterAvailabilityZoneMixin._process_az_request of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73e77bf35160>> router precommit_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.084862 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_create of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73e77bf35160>> router precommit_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.084947 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_update of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73e77bf35160>> router precommit_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.085034 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <function OVNL3RouterPlugin._subnet_update at 0x73e77bf80860> subnet after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.085214 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.quota.resource_registry [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Tracking information for resource: router configured {{(pid=91387) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 27 15:57:37.085302 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.quota.resource_registry [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Tracking information for resource: floatingip configured {{(pid=91387) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 27 15:57:37.085392 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.services.ovn_l3.plugin [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Starting OVNL3RouterPlugin
May 27 15:57:37.085585 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNPortForwarding.register of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73e77bf35610>> port_forwarding_plugin after_init 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.085813 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method DriverController._check_router_request of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x73e77bf36a50>> router before_create 55550100, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.085896 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method DriverController._clear_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x73e77bf36a50>> router precommit_delete 55550100, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.085977 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method DriverController._set_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x73e77bf36a50>> router precommit_create 55550100, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.086056 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method DriverController._update_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x73e77bf36a50>> router precommit_update 55550100, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.088211 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.services.provider_configuration [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Service providers = [] {{(pid=91387) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}}
May 27 15:57:37.088303 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.db.servicetype_db [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=91387) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:45}}
May 27 15:57:37.089701 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVN_L3_HA_db_mixin._precommit_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> router precommit_create 55549900, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.089854 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_add_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> router_interface after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.089901 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_floatingip_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> floatingip after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.090079 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> floatingip precommit_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.090079 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> floatingip precommit_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.090246 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_floatingip_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> floatingip after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.090246 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_floatingip_status_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> floatingip after_status_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.090394 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_floatingip_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> floatingip after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.090437 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_portforwarding_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> port_forwarding after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.090576 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_remove_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> router_interface after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.090682 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> router after_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.090804 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_router_create_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> router precommit_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.090907 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_router_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> router after_delete 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.091009 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OvnDriver._process_router_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73e77bf78950>> router after_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.091115 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.services.service_base [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=91387) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}}
May 27 15:57:37.091261 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <bound method OVNL3RouterPlugin._post_fork_initialize of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73e77bf35160>> process after_init 55550001, True {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.091404 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=91387) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 27 15:57:37.091506 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading Plugin: auto_allocate
May 27 15:57:37.092389 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x73e77bf82980> network precommit_update 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.092497 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron_lib.callbacks.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x73e77bf82980> network precommit_create 55550000, False {{(pid=91387) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.092683 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=91387) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 27 15:57:37.092777 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading Plugin: tag
May 27 15:57:37.095710 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=91387) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 27 15:57:37.095710 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading Plugin: timestamp
May 27 15:57:37.096778 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=91387) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 27 15:57:37.096915 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading Plugin: network_ip_availability
May 27 15:57:37.098434 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=91387) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 27 15:57:37.098550 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading Plugin: flavors
May 27 15:57:37.099726 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=91387) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 27 15:57:37.099838 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading Plugin: revisions
May 27 15:57:37.101005 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.manager [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=91387) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 27 15:57:37.101172 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG oslo_concurrency.lockutils [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.583s {{(pid=91387) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:543}}
May 27 15:57:37.101402 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=91387) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}}
May 27 15:57:37.101509 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Initializing extension manager.
May 27 15:57:37.101822 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: __init__.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.101916 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: __pycache__ {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.101963 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: aap_reject_multicast.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.102310 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Allowed address pairs reject multicast extension" alias="aap-reject-multicast" description="Reject multicast addresses in Allowed Address Pairs" updated="2026-04-01T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.102673 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension aap-reject-multicast not supported by any of loaded plugins
May 27 15:57:37.102852 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: address_group.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.103237 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.103577 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: address-group
May 27 15:57:37.103577 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: address_scope.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.103908 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.104056 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: address-scope
May 27 15:57:37.104168 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: admin_state_down_before_update.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.104469 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.104679 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins
May 27 15:57:37.104787 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: agent.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.105130 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.105300 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: agent
May 27 15:57:37.105413 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: agent_resources_synced.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.105688 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.105822 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension agent-resources-synced not supported by any of loaded plugins
May 27 15:57:37.105939 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: agent_sort_key.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.106123 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Agents Sort Key" alias="agent_sort_key" description="Enabling the keyword sort_key for sorting in agents." updated="2013-02-03T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.106298 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension agent_sort_key not supported by any of loaded plugins
May 27 15:57:37.106375 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: allowedaddresspairs.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.107131 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.107360 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: allowed-address-pairs
May 27 15:57:37.107468 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: auto_allocated_topology.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.107800 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.108006 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: auto-allocated-topology
May 27 15:57:37.108130 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: availability_zone.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.108477 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.108698 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: availability_zone
May 27 15:57:37.108807 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: availability_zone_filter.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.109072 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.109265 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension availability_zone_filter not supported by any of loaded plugins
May 27 15:57:37.109404 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: data_plane_status.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.109682 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.109879 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension data-plane-status not supported by any of loaded plugins
May 27 15:57:37.110001 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: default_subnetpools.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.110264 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.110520 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: default-subnetpools
May 27 15:57:37.110649 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: dhcpagentscheduler.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.111081 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.111265 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: dhcp_agent_scheduler
May 27 15:57:37.111406 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: dns.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.111665 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.111861 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: dns-integration
May 27 15:57:37.111969 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: dns_domain_ports.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.112248 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.112452 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: dns-domain-ports
May 27 15:57:37.112566 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.112837 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like <project_id>, <project_name>, <user_id> and <user_name> as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.113017 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: dns-integration-domain-keywords
May 27 15:57:37.113124 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: dvr.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.113441 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.113689 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension dvr not supported by any of loaded plugins
May 27 15:57:37.113744 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: ecmp_routes.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.114013 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.114198 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension ecmp_routes not supported by any of loaded plugins
May 27 15:57:37.114301 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: empty_string_filtering.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.114572 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.114771 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: empty-string-filtering
May 27 15:57:37.114885 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: evpn.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.115127 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="EVPN" alias="evpn" description="Router extension to support EVPN Type-5 route advertisement" updated="2026-04-28T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.115309 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension evpn not supported by any of loaded plugins
May 27 15:57:37.115432 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.115702 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.115885 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins
May 27 15:57:37.115988 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.116236 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.116444 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins
May 27 15:57:37.116563 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: external_net.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.116843 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.117001 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: external-net
May 27 15:57:37.117114 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: extra_dhcp_opt.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.117375 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.117558 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: extra_dhcp_opt
May 27 15:57:37.117686 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: extraroute.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.117947 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.118136 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: extraroute
May 27 15:57:37.118254 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: extraroute_atomic.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.118540 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.118778 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: extraroute-atomic
May 27 15:57:37.118855 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: filter_validation.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.119130 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.119309 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: filter-validation
May 27 15:57:37.119453 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: fip_pf_description.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.119719 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.120120 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins
May 27 15:57:37.120240 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: fip_pf_detail.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.120519 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.120761 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: floating-ip-port-forwarding-detail
May 27 15:57:37.120869 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: fip_pf_port_range.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.121142 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.121415 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins
May 27 15:57:37.121504 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: fip_port_details.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.121767 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.121956 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: fip-port-details
May 27 15:57:37.122069 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: flavors.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.122385 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.122565 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: flavors
May 27 15:57:37.122688 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.123062 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.123249 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins
May 27 15:57:37.123372 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: floatingip_pools.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.123666 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.123886 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: floatingip-pools
May 27 15:57:37.123963 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: ip_allocation.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.124216 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.124417 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension ip_allocation not supported by any of loaded plugins
May 27 15:57:37.124535 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.124807 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.124982 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension ip-substring-filtering not supported by any of loaded plugins
May 27 15:57:37.125094 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l2_adjacency.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.125351 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.125541 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension l2_adjacency not supported by any of loaded plugins
May 27 15:57:37.125669 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.126156 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.126383 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: router
May 27 15:57:37.126450 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3_conntrack_helper.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.127138 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.127347 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension l3-conntrack-helper not supported by any of loaded plugins
May 27 15:57:37.127463 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.127752 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.127982 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: enable-default-route-bfd
May 27 15:57:37.128077 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.128368 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.128547 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: enable-default-route-ecmp
May 27 15:57:37.128679 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.128932 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.129130 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: ext-gw-mode
May 27 15:57:37.129249 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.129508 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.129705 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: l3-ha
May 27 15:57:37.129825 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.130090 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.130292 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins
May 27 15:57:37.130417 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3_extra_gws.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.130680 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.130863 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: external-gateway-multihoming
May 27 15:57:37.130977 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3_flavors.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.131236 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.131455 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: l3-flavors
May 27 15:57:37.131558 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3_ndp_proxy.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.131910 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.132087 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension l3-ndp-proxy not supported by any of loaded plugins
May 27 15:57:37.132206 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.132512 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.132730 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins
May 27 15:57:37.132835 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: l3agentscheduler.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.133391 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.133590 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension l3_agent_scheduler not supported by any of loaded plugins
May 27 15:57:37.133745 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: local_ip.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.134101 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.134309 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension local_ip not supported by any of loaded plugins
May 27 15:57:37.134427 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: logging.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.134783 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.134969 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension logging not supported by any of loaded plugins
May 27 15:57:37.135081 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: metering.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.135601 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.135848 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension metering not supported by any of loaded plugins
May 27 15:57:37.135971 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.136242 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.136441 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins
May 27 15:57:37.136555 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: multiprovidernet.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.136808 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.137004 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: multi-provider
May 27 15:57:37.137157 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: netmtu.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.137386 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.137559 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: net-mtu
May 27 15:57:37.137683 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: netmtu_writable.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.137936 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.138119 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: net-mtu-writable
May 27 15:57:37.138223 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: network_availability_zone.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.138532 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.138729 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: network_availability_zone
May 27 15:57:37.138851 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: network_ha.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.139093 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.139276 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension network_ha not supported by any of loaded plugins
May 27 15:57:37.139415 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: network_ip_availability.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.139702 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.139892 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: network-ip-availability
May 27 15:57:37.140015 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: network_ip_availability_details.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.140284 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.140479 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: network-ip-availability-details
May 27 15:57:37.140583 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: network_segment_range.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.141054 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.141242 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension network-segment-range not supported by any of loaded plugins
May 27 15:57:37.141388 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: pagination.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.141640 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.141758 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: pagination
May 27 15:57:37.141866 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_device_profile.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.142113 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.142310 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: port-device-profile
May 27 15:57:37.142426 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_hardware_offload_type.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.142697 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.142909 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension port-hardware-offload-type not supported by any of loaded plugins
May 27 15:57:37.143012 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.143274 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.143483 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins
May 27 15:57:37.143600 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_hints.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.143858 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.144042 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension port-hints not supported by any of loaded plugins
May 27 15:57:37.144269 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_mac_address_override.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.144433 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.144661 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension port-mac-override not supported by any of loaded plugins
May 27 15:57:37.144785 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.145054 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.145267 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: port-mac-address-regenerate
May 27 15:57:37.145383 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.145677 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.145859 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: port-numa-affinity-policy
May 27 15:57:37.145965 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.146219 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.146414 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: port-numa-affinity-policy-socket
May 27 15:57:37.146512 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_resource_request.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.146777 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.146956 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension port-resource-request not supported by any of loaded plugins
May 27 15:57:37.147061 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_resource_request_groups.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.147347 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.147532 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension port-resource-request-groups not supported by any of loaded plugins
May 27 15:57:37.147650 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: port_trusted_vif.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.147901 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.148079 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension port-trusted-vif not supported by any of loaded plugins
May 27 15:57:37.148188 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: portbindings.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.148446 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.148635 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: binding
May 27 15:57:37.148768 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: portbindings_extended.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.149070 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.149251 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: binding-extended
May 27 15:57:37.149367 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: portsecurity.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.149630 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.149812 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: port-security
May 27 15:57:37.149926 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: project_id.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.150175 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.150299 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: project-id
May 27 15:57:37.150415 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: providernet.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.150719 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.150876 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: provider
May 27 15:57:37.150991 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: pvlan.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.151237 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Private VLAN" alias="pvlan" description="The Private VLAN semantics extension." updated="2026-02-03T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.151429 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension pvlan not supported by any of loaded plugins
May 27 15:57:37.151532 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qinq.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.151829 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x88a8 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.152022 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: qinq
May 27 15:57:37.152140 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.152944 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.153147 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos not supported by any of loaded plugins
May 27 15:57:37.153273 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.153612 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.153811 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins
May 27 15:57:37.153927 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.154175 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.154427 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins
May 27 15:57:37.154539 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_default.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.154825 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.155011 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-default not supported by any of loaded plugins
May 27 15:57:37.155118 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_fip.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.155379 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.155701 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-fip not supported by any of loaded plugins
May 27 15:57:37.155822 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_fip_network_policy.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.156103 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.156287 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-fip-network-policy not supported by any of loaded plugins
May 27 15:57:37.156418 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_gateway_ip.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.156682 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.156865 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-gateway-ip not supported by any of loaded plugins
May 27 15:57:37.156975 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_port_network_policy.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.157223 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.157416 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-port-network-policy not supported by any of loaded plugins
May 27 15:57:37.157544 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.157831 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.158016 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-pps-minimum not supported by any of loaded plugins
May 27 15:57:37.158124 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.158400 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.158706 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins
May 27 15:57:37.158859 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_pps_rule.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.159159 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.159353 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-pps not supported by any of loaded plugins
May 27 15:57:37.159462 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_rule_type_details.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.159729 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.159911 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-rule-type-details not supported by any of loaded plugins
May 27 15:57:37.160039 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_rule_type_filter.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.160295 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.160492 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-rule-type-filter not supported by any of loaded plugins
May 27 15:57:37.160643 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: qos_rules_alias.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.160977 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.161176 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension qos-rules-alias not supported by any of loaded plugins
May 27 15:57:37.161298 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: quota_check_limit.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.161580 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.161789 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: quota-check-limit
May 27 15:57:37.161896 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: quota_check_limit_default.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.162166 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.162360 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: quota-check-limit-default
May 27 15:57:37.162473 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: quotasv2.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.163049 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.163189 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: quotas
May 27 15:57:37.163261 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: quotasv2_detail.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.163924 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.164043 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: quota_details
May 27 15:57:37.164155 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: rbac.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.164750 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control project access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.164991 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: rbac-policies
May 27 15:57:37.165126 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: rbac_address_group.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.165377 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.165566 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension rbac-address-group not supported by any of loaded plugins
May 27 15:57:37.165695 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: rbac_address_scope.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.165949 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.166134 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: rbac-address-scope
May 27 15:57:37.166288 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: rbac_security_groups.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.166505 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.166701 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: rbac-security-groups
May 27 15:57:37.166804 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: rbac_subnetpool.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.167054 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.167238 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension rbac-subnetpool not supported by any of loaded plugins
May 27 15:57:37.167354 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: revisionifmatch.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.167639 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.167828 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: revision-if-match
May 27 15:57:37.167939 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: revisions.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.168226 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.168422 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: standard-attr-revisions
May 27 15:57:37.168531 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: router_availability_zone.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.168838 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.169024 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: router_availability_zone
May 27 15:57:37.169130 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: router_enable_snat.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.169394 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.169580 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: router-enable-snat
May 27 15:57:37.169704 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: routerservicetype.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.169948 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.170129 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension router-service-type not supported by any of loaded plugins
May 27 15:57:37.170235 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: security_groups_default_rules.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.170743 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.170938 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: security-groups-default-rules
May 27 15:57:37.171055 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.171296 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.171496 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: security-groups-normalized-cidr
May 27 15:57:37.171613 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: security_groups_port_filtering.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.171884 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.172064 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension port-security-groups-filtering not supported by any of loaded plugins
May 27 15:57:37.172180 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.172431 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.172616 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: security-groups-remote-address-group
May 27 15:57:37.172742 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.173000 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.173185 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: security-groups-rules-belongs-to-default-sg
May 27 15:57:37.173287 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: security_groups_shared_filtering.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.173562 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.173761 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: security-groups-shared-filtering
May 27 15:57:37.173883 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: securitygroup.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.175460 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.175702 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: security-group
May 27 15:57:37.175767 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: segment.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.176072 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.176257 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension segment not supported by any of loaded plugins
May 27 15:57:37.176356 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.176560 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.176696 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins
May 27 15:57:37.176779 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: servicetype.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.176970 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.177091 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: service-type
May 27 15:57:37.177177 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: sorting.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.177359 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.177461 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: sorting
May 27 15:57:37.177606 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: standard_attr_segment.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.177770 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.177893 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension standard-attr-segment not supported by any of loaded plugins
May 27 15:57:37.177968 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: standardattrdescription.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.178214 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.178404 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: standard-attr-description
May 27 15:57:37.178539 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: stateful_security_group.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.178802 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.178987 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: stateful-security-group
May 27 15:57:37.179093 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: stdattrs_common.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.179344 np0000001523 neutron-ovn-maintenance-worker[91387]: WARNING neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py
May 27 15:57:37.179450 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.179717 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.179906 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: subnet-dns-publish-fixed-ip
May 27 15:57:37.180009 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: subnet_external_network.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.180261 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.180453 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: subnet-external-network
May 27 15:57:37.180562 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: subnet_onboard.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.180851 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.181035 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension subnet_onboard not supported by any of loaded plugins
May 27 15:57:37.181149 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.181405 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.181588 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins
May 27 15:57:37.181718 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: subnet_service_types.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.182263 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.182494 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: subnet-service-types
May 27 15:57:37.182634 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: subnetallocation.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.182924 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.183132 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: subnet_allocation
May 27 15:57:37.183251 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.183582 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.183796 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: subnetpool-prefix-ops
May 27 15:57:37.183916 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: tag_creation.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.184182 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.184385 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: tag-creation
May 27 15:57:37.184506 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.184876 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.184991 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins
May 27 15:57:37.185100 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: tagging.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.185964 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.186188 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: standard-attr-tag
May 27 15:57:37.186269 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: timestamp.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.186570 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.186775 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: standard-attr-timestamp
May 27 15:57:37.186881 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: trunk.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.187160 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.187374 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension trunk not supported by any of loaded plugins
May 27 15:57:37.187482 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: trunk_details.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.187752 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.187940 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension trunk-details not supported by any of loaded plugins
May 27 15:57:37.188068 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: uplink_status_propagation.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.188316 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.188506 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension uplink-status-propagation not supported by any of loaded plugins
May 27 15:57:37.188614 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.188887 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.189069 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins
May 27 15:57:37.189164 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loading extension file: vlantransparent.py {{(pid=91387) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 27 15:57:37.189467 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=91387) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 27 15:57:37.189667 np0000001523 neutron-ovn-maintenance-worker[91387]: INFO neutron.api.extensions [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Loaded extension: vlan-transparent
May 27 15:57:37.193693 np0000001523 neutron-ovn-maintenance-worker[91387]: WARNING oslo_service.backend._threading.service [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback.
May 27 15:57:37.194074 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Full set of CONF: {{(pid=91387) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}}
May 27 15:57:37.194216 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ******************************************************************************** {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3229}}
May 27 15:57:37.194292 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Configuration options gathered from: {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3230}}
May 27 15:57:37.194385 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3231}}
May 27 15:57:37.194563 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3232}}
May 27 15:57:37.194683 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ================================================================================ {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3237}}
May 27 15:57:37.194858 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] agent_down_time                = 75 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.194996 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] allow_automatic_dhcp_failover  = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.195187 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] allow_bulk                     = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.195399 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.195539 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] api_extensions_path            =  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.195700 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] api_paste_config               = api-paste.ini {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.195849 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] api_workers                    = 3 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.195987 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] auth_strategy                  = keystone {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.196128 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.196290 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] bind_host                      = 0.0.0.0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.196437 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] bind_port                      = 9696 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.196575 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] client_socket_timeout          = 900 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.196742 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] config_dir                     = [] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.196848 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.196983 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] config_source                  = [] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.197127 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] control_exchange               = neutron {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.197269 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] core_plugin                    = ml2 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.197418 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] debug                          = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.197554 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] default_availability_zones     = [] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.197800 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.197971 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] dhcp_agent_notification        = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.198084 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] dhcp_agents_per_network        = 1 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.198220 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] dhcp_lease_duration            = 86400 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.198372 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] dhcp_load_type                 = networks {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.198513 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] dns_domain                     = openstacklocal {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.198669 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.198805 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] enable_default_route_bfd       = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.198942 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] enable_default_route_ecmp      = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.199073 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] enable_dvr                     = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.199201 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] enable_new_agents              = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.199328 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] enable_services_on_agents_with_admin_state_down = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.199469 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] enable_snat_by_default         = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.199599 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] enable_traditional_dhcp        = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.199741 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] external_dns_driver            = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.199868 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] filter_validation              = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.200006 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] global_physnet_mtu             = 1430 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.200118 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] graceful_shutdown_timeout      = 60 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.200307 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] host                           = np0000001523 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.200452 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] host_dvr_for_dhcp              = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.200583 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] http_retries                   = 3 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.200740 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] instance_format                = [instance: %(uuid)s]  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.200876 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.201012 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ipam_driver                    = internal {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.201141 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ipv6_pd_enabled                = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.201272 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] l3_ha                          = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.201417 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.201552 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] l3_ha_network_physical_name    =  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.201707 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] l3_ha_network_type             =  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.201844 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] log_color                      = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.201979 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] log_config_append              = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.202115 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.202242 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] log_dir                        = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.202381 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] log_file                       = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.202483 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] log_options                    = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.202615 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] log_rotate_interval            = 1 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.202773 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] log_rotate_interval_type       = days {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.202911 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] log_rotation_type              = none {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.203015 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.203125 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.203255 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.203402 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.203502 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.203670 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] max_allowed_address_pair       = 10 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.203807 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] max_dns_nameservers            = 5 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.203934 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] max_header_line                = 16384 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.204071 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] max_l3_agents_per_router       = 3 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.204195 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] max_logfile_count              = 30 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.204326 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] max_logfile_size_mb            = 200 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.204464 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] max_routes                     = 30 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.204595 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] max_subnet_host_routes         = 20 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.204744 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] my_ip                          = 10.4.3.150 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.204877 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] my_ipv6                        = ::1 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.205006 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] network_auto_schedule          = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.205131 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] network_link_prefix            = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.205329 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.205432 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] notify_nova_on_port_data_changes = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.205559 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] notify_nova_on_port_status_changes = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.205713 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] pagination_max_limit           = -1 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.205841 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] periodic_fuzzy_delay           = 5 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.205973 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] periodic_interval              = 40 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.206109 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] publish_errors                 = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.206243 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] rate_limit_burst               = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.206390 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] rate_limit_except_level        = CRITICAL {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.206517 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] rate_limit_interval            = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.206673 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] router_distributed             = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.206805 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] rpc_resources_processing_step  = 20 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.206934 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] rpc_response_max_timeout       = 600 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.207064 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] rpc_response_timeout           = 60 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.207199 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] rpc_state_report_workers       = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.207326 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] rpc_workers                    = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.207467 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] send_events_interval           = 2 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.207611 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] service_plugins                = ['ovn-router'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.207766 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] setproctitle                   = on {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.207899 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] shell_completion               = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.208045 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] state_path                     = /opt/stack/data/neutron {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.208187 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] syslog_log_facility            = LOG_USER {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.208317 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] tcp_keepidle                   = 600 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.208467 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] transport_url                  = **** {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.208603 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] use_journal                    = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.208754 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] use_json                       = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.208889 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] use_stderr                     = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.209037 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] use_syslog                     = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.209171 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] vlan_qinq                      = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.209300 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] vlan_transparent               = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.209448 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] wsgi_default_pool_size         = 100 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.209580 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] wsgi_keep_alive                = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.209729 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.209857 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] wsgi_server_debug              = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.210007 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_concurrency.disable_process_locking = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.210147 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.210303 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.connection_string     = messaging:// {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.210449 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.enabled               = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.210589 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.es_doc_type           = notification {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.210739 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.es_scroll_size        = 10000 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.210876 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.es_scroll_time        = 2m {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.211007 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.filter_error_trace    = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.211142 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.hmac_keys             = **** {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.211276 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.sentinel_service_name = mymaster {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.211430 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.socket_timeout        = 0.1 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.211563 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.trace_requests        = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.211712 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler.trace_sqlalchemy      = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.211856 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler_jaeger.process_tags   = {} {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.211985 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler_jaeger.service_name_prefix = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.212116 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] profiler_otlp.service_name_prefix = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.212250 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.212403 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.212548 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.212709 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.212841 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.212996 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.enforce_new_defaults = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.213276 np0000001523 neutron-ovn-maintenance-worker[91387]: WARNING oslo_config.cfg [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.).  Its value may be silently ignored in the future.
May 27 15:57:37.213430 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.enforce_scope      = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.213569 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.policy_default_rule = default {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.213720 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.213868 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.214009 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.214139 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.214274 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.214413 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.214543 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.214691 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_policy.remote_timeout     = 60.0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.214835 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.214979 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] service_providers.service_provider = [] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.215118 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.auth_section              = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.215257 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.auth_type                 = password {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.215425 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.auth_url                  = https://10.4.3.150/identity {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.215548 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.cafile                    = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.215703 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.certfile                  = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.215833 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.collect_timing            = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.215965 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.default_domain_id         = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.216094 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.default_domain_name       = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.216224 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.domain_id                 = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.216381 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.domain_name               = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.216523 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.endpoint_type             = public {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.216669 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.insecure                  = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.216797 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.keyfile                   = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.216932 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.password                  = **** {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.217060 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.project_domain_id         = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.217197 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.project_domain_name       = Default {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.217329 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.project_id                = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.217480 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.project_name              = service {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.217674 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.region_name               = RegionOne {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.217811 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.split_loggers             = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.217958 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.system_scope              = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.218084 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.timeout                   = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.218220 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.trust_id                  = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.218356 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.user_domain_id            = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.218501 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.user_domain_name          = Default {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.218641 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.user_id                   = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.218790 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] nova.username                  = nova {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.218932 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.auth_section         = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.219076 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.auth_type            = password {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.219205 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.cafile               = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.219342 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.certfile             = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.219475 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.collect_timing       = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.219614 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.endpoint_type        = public {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.219759 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.insecure             = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.219894 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.keyfile              = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.220032 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.region_name          = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.220172 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.split_loggers        = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.220298 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] placement.timeout              = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.220457 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.auth_section            = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.220585 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.auth_type               = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.220736 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.cafile                  = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.220868 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.certfile                = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.221010 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.collect_timing          = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.221139 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.connect_retries         = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.221267 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.connect_retry_delay     = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.221410 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.enable_notifications    = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.221538 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.endpoint_override       = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.221686 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.insecure                = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.221811 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.interface               = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.221938 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.keyfile                 = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.222067 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.max_version             = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.222193 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.min_version             = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.222325 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.region_name             = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.222467 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.retriable_status_codes  = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.222665 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.service_name            = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.222795 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.service_type            = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.222926 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.split_loggers           = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.223057 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.status_code_retries     = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.223183 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.status_code_retry_delay = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.223309 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.timeout                 = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.223446 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.valid_interfaces        = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.223574 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ironic.version                 = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.223730 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] cli_script.dry_run             = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.223878 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.224011 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.224145 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.224276 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.224417 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.224552 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.224700 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.224833 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.224980 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.hostname = np0000001523 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.225108 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.225241 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.225389 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.225556 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.225693 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.225829 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.225970 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.226100 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.226241 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.226386 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.226521 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.226671 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.226808 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.226952 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.227089 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.227231 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.227377 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.227518 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.227657 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.227786 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.227927 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.ssl      = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.228083 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.228237 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.228379 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.228517 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.ssl_enforce_hostname_verification = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.228674 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.228808 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.228947 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.229084 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_notifications.driver = [] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.229216 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_notifications.retry = -1 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.229370 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.229499 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_messaging_notifications.transport_url = **** {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.229652 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_reports.file_event_handler = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.229788 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.229926 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_reports.log_dir           = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.230060 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.230200 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.default_quota           = -1 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.230355 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.230494 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.quota_floatingip        = 50 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.230641 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.quota_network           = 100 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.230769 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.quota_port              = 500 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.230901 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.231034 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.quota_router            = 10 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.231169 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.quota_security_group    = 10 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.231296 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.quota_security_group_rule = 100 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.231436 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.quota_subnet            = 100 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.231568 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] QUOTAS.track_quota_usage       = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.231725 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.231853 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep.group                  = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.231980 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep.helper_command         = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.232112 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep.log_daemon_traceback   = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.232247 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.232388 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep.thread_pool_size       = 12 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.232516 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep.user                   = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.232670 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.232799 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_dhcp_release.group     = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.232932 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_dhcp_release.helper_command = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.233066 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.233212 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.233351 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_dhcp_release.thread_pool_size = 12 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.233488 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_dhcp_release.user      = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.233635 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.233769 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_ovs_vsctl.group        = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.233905 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_ovs_vsctl.helper_command = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.234042 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.234172 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.234316 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_ovs_vsctl.thread_pool_size = 12 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.234454 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_ovs_vsctl.user         = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.234595 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_namespace.capabilities = [21] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.234735 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_namespace.group        = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.234864 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_namespace.helper_command = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.234991 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_namespace.log_daemon_traceback = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.235121 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.235255 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_namespace.thread_pool_size = 12 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.235390 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_namespace.user         = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.235521 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_conntrack.capabilities = [12] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.235691 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_conntrack.group        = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.235818 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_conntrack.helper_command = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.235930 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_conntrack.log_daemon_traceback = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.236067 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.236204 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_conntrack.thread_pool_size = 12 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.236343 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_conntrack.user         = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.236482 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_link.capabilities      = [12, 21] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.236618 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_link.group             = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.236766 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_link.helper_command    = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.236915 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_link.log_daemon_traceback = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.237041 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.237176 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_link.thread_pool_size  = 12 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.237305 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] privsep_link.user              = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.237468 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.237609 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.237762 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] AGENT.use_helper_for_ns_read   = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.237900 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] SECURITYGROUP.enable_ipset     = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.238051 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] SECURITYGROUP.enable_security_group = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.238213 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] SECURITYGROUP.firewall_driver  = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.238363 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.238512 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] experimental.ipv6_pd_enabled   = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.238693 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.238824 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2.external_network_type      = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.238973 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2.mechanism_drivers          = ['ovn'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.239111 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2.overlay_ip_version         = 4 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.239245 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2.path_mtu                   = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.239391 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2.physical_network_mtus      = {} {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.239535 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2.project_network_types      = ['geneve'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.239687 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.239844 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.239995 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2_type_flat.flat_networks    = ['public'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.240135 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2_type_geneve.max_header_size = 38 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.240274 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.240437 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.240579 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] network_log.burst_limit        = 25 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.240720 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] network_log.local_output_log_base = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.240858 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] network_log.rate_limit         = 100 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.240999 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.asyncio_connection    = **** {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.241134 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.asyncio_slave_connection = **** {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.241276 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.backend               = sqlalchemy {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.241422 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.connection            = **** {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.241554 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.connection_debug      = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.241713 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.connection_parameters =  {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.241843 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.connection_recycle_time = 3600 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.241969 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.connection_trace      = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.242103 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.db_inc_retry_interval = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.242231 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.db_max_retries        = 20 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.242369 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.db_max_retry_interval = 10 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.242500 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.db_retry_interval     = 1 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.242635 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.max_overflow          = 50 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.242780 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.max_pool_size         = 5 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.242915 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.max_retries           = 10 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.243052 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.243188 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.mysql_wsrep_sync_wait = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.243313 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.pool_timeout          = None {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.243460 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.retry_interval        = 10 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.243596 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.slave_connection      = **** {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.243757 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.sqlite_synchronous    = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.243896 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.synchronous_reader    = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.244029 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] database.use_db_reconnect      = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.244168 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.bfd_min_rx                 = 1000 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.244304 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.bfd_min_tx                 = 100 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.244445 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.bfd_mult                   = 3 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.244577 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.broadcast_arps_to_all_routers = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.244736 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.dhcp_default_lease_time    = 43200 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.244868 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.244998 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.dns_records_ovn_owned      = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.245130 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.dns_servers                = [] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.245264 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.enable_distributed_floating_ip = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.245410 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.fdb_age_threshold          = 300 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.245556 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ha_failover_strategy       = normal {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.245713 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.live_migration_activation_strategy = rarp {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.245900 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.localnet_learn_fdb         = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.245988 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.mac_binding_age_threshold  = 300 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.246136 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.neutron_sync_mode          = log {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.246269 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_dhcp4_global_options   = {} {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.246413 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_dhcp6_global_options   = {} {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.246562 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_l3_scheduler           = leastloaded {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.246724 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_metadata_enabled       = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.246862 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.247006 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.247160 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_nb_connection          = ['ssl:10.4.3.150:6641'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.247296 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.247446 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_router_indirect_snat   = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.247587 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.247743 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.247885 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_sb_connection          = ['ssl:10.4.3.150:6642'] {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.248023 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.248156 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovs_create_tap             = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.248292 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovsdb_connection_timeout   = 180 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.248444 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovsdb_log_level            = INFO {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.248573 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovsdb_probe_interval       = 60000 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.248722 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.ovsdb_retry_max_interval   = 180 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.248855 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.stateless_nat_enabled      = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.248995 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.249139 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] OVS.bridge_mac_table_size      = 50000 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.249274 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] OVS.igmp_flood                 = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.249405 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] OVS.igmp_flood_reports         = True {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.249548 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] OVS.igmp_flood_unregistered    = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.249702 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] OVS.igmp_snooping_enable       = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.249834 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] OVS.ovsdb_timeout              = 10 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.249970 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.250100 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn_nb_global.ignore_lsp_down  = False {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.250233 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.250348 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG cotyledon.oslo_config_glue [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] ******************************************************************************** {{(pid=91387) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3263}}
May 27 15:57:37.250483 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG oslo_service.backend._threading.service [None req-78f78384-81fc-4939-989a-e0045c786b72 None None] Selected the multiprocessing context: <multiprocessing.context.ForkContext object at 0x73e7811cce60> and updated it in Cotyledon ServiceManager: <multiprocessing.context.ForkContext object at 0x73e7811cce60> {{(pid=91387) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:405}}
May 27 15:57:37.271793 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=92285) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}}
May 27 15:57:37.272443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3229}}
May 27 15:57:37.272443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3230}}
May 27 15:57:37.272443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3231}}
May 27 15:57:37.272443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3232}}
May 27 15:57:37.272443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3237}}
May 27 15:57:37.272566 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time                = 75 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.272750 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover  = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.272822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk                     = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.273093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.273209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path            =  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.273319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config               = api-paste.ini {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.273460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] api_workers                    = 3 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.273580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy                  = keystone {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.273710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] base_mac                       = fa:16:3e:00:00:00 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.273973 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] bind_host                      = 0.0.0.0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.274089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] bind_port                      = 9696 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.274207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout          = 900 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.274346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] config_dir                     = [] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.274420 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.275780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] config_source                  = [] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.275933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] control_exchange               = neutron {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.276045 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] core_plugin                    = ml2 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.276230 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] debug                          = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.276305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones     = [] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.276530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.276667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification        = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.276781 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network        = 1 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.276902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration            = 86400 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.277024 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type                 = networks {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.277126 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] dns_domain                     = openstacklocal {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.277251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.277372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd       = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.277465 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp      = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.277578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr                     = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.277698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents              = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.277796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.277896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default         = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.277995 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp        = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.278082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver            = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.278142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations             = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.278260 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] filter_validation              = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.278379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu             = 1430 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.278468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout      = 60 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.278750 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] host                           = np0000001523 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.278869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp              = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.278956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] http_retries                   = 3 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.279047 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] instance_format                = [instance: %(uuid)s]  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.279157 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.279248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver                    = internal {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.279366 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled                = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] l3_ha                          = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.279646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.279758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name    =  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.279861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type             =  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.279981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] log_color                      = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.280095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] log_config_append              = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.280207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.280313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] log_dir                        = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.280427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] log_file                       = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.280497 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] log_options                    = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.280594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval            = 1 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.280723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type       = days {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.280819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type              = none {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.280912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.280974 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.281039 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.281145 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.281326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair       = 10 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.281444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers            = 5 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] max_header_line                = 16384 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.281658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router       = 3 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.281744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count              = 30 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.281845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb            = 200 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.281935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] max_routes                     = 30 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.282029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes         = 20 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.282135 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] my_ip                          = 10.4.3.150 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.282263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6                        = ::1 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.282365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule          = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.282454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix            = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.282567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.282687 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.282802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.282905 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit           = -1 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.283009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay           = 5 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.283117 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval              = 40 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.283198 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] publish_errors                 = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.283280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst               = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.283370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level        = CRITICAL {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.283487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval            = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.283592 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] router_distributed             = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step  = 20 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.283830 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout       = 600 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.283939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout           = 60 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.284027 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers       = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.284094 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers                    = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval           = 2 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.284320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] service_plugins                = ['ovn-router'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.284420 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] setproctitle                   = on {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] shell_completion               = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.284616 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] state_path                     = /opt/stack/data/neutron {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.284740 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility            = LOG_USER {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.284833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle                   = 600 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.284934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] transport_url                  = **** {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.285013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] use_journal                    = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.285086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] use_json                       = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.285180 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] use_stderr                     = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.285277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] use_syslog                     = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.285385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq                      = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.285480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent               = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.285567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size         = 100 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.285707 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive                = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.285866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.285952 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug              = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}}
May 27 15:57:37.286118 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.286273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.286417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string     = messaging:// {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.286526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled               = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.286614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type           = notification {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.286736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size        = 10000 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.286821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time        = 2m {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.286916 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace    = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.287051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys             = **** {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.287125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.287246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout        = 0.1 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.287341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests        = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.287443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy      = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.287578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags   = {} {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.287699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.287801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.287925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name =  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288323 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288549 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope      = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288846 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.288945 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289157 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout     = 60.0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289243 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section              = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type                 = password {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289508 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url                  = https://10.4.3.150/identity {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile                    = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile                  = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing            = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id         = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name       = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id                 = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.289913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name               = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290002 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type             = public {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure                  = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile                   = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.password                  = **** {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id         = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name       = Default {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id                = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name              = service {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name               = RegionOne {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290545 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers             = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope              = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout                   = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290735 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id                  = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290792 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id            = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290841 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name          = Default {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id                   = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] nova.username                  = nova {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.290994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section         = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291047 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type            = password {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291117 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile               = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile             = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing       = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type        = public {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure             = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile              = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name          = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers        = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout              = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section            = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type               = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile                  = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291778 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile                = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing          = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries         = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.291928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay     = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications    = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override       = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure                = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface               = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile                 = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version             = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version             = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name             = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes  = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292462 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name            = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type            = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292559 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers           = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries     = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292756 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout                 = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292756 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces        = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ironic.version                 = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run             = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.292993 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293062 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293306 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000001523 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293732 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293785 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293898 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.293984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294131 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294180 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl      = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294905 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294949 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_hostname_verification = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.294979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file =  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295045 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version =  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295442 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295503 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir           = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295743 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota           = -1 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip        = 50 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network           = 100 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.295990 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port              = 500 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy       = 10 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router            = 10 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296157 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group    = 10 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet            = 100 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage       = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep.group                  = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command         = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback   = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name            = oslo_privsep.daemon {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296746 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size       = 12 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296828 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep.user                   = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296828 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group     = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.296994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297038 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297068 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 12 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user      = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group        = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297386 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 12 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user         = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297573 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group        = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297772 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 12 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.297942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user         = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group        = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298112 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298299 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 12 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user         = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities      = [12, 21] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group             = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command    = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size  = 12 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298733 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user              = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298839 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.298919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read   = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset     = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver  = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299282 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled   = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type      = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers          = ['ovn'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version         = 4 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu                   = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299714 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus      = {} {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types      = ['geneve'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.299975 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks    = ['public'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300039 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300187 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit        = 25 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300419 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit         = 100 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300479 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection    = **** {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300549 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300592 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.backend               = sqlalchemy {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.connection            = **** {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300749 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug      = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters =  {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300887 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace      = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.300997 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries        = 20 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval     = 1 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow          = 50 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size         = 5 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries           = 10 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode        = TRADITIONAL {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301437 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout          = None {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval        = 10 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection      = **** {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous    = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader    = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect      = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx                 = 1000 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx                 = 100 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult                   = 3 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.301986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time    = 43200 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302199 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned      = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302199 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers                = [] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold          = 300 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy       = normal {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb         = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold  = 300 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode          = log {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302663 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options   = {} {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options   = {} {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler           = leastloaded {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled       = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.302944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection          = ['ssl:10.4.3.150:6641'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303159 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat   = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection          = ['ssl:10.4.3.150:6642'] {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303350 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303425 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap             = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout   = 180 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level            = INFO {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval       = 60000 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval   = 180 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled      = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303781 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size      = 50000 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood                 = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.303967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports         = True {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.304008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered    = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.304080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable       = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.304162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout              = 10 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.304232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.304272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down  = False {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.304356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}}
May 27 15:57:37.304418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=92285) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3263}}
May 27 15:57:37.304607 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [92285] {{(pid=92285) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}}
May 27 15:57:37.305568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-892796', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-28332'] for process (None), after_init {{(pid=92285) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 27 15:57:37.307041 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry
May 27 15:57:37.352040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 27 15:57:37.352157 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 27 15:57:37.352273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 27 15:57:37.352273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352394 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352394 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352465 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352738 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352738 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352738 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.352821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.356079 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:10.4.3.150:6641: connecting...
May 27 15:57:37.361914 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:10.4.3.150:6641: connected
May 27 15:57:37.379879 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry
May 27 15:57:37.406450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 27 15:57:37.406548 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 27 15:57:37.406606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.406708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=92285) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 27 15:57:37.409635 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:10.4.3.150:6642: connecting...
May 27 15:57:37.417097 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:10.4.3.150:6642: connected
May 27 15:57:37.454459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73e77b29e9c0>> floatingip precommit_update 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.454589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73e77b29e9c0>> floatingip precommit_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.454794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73e77b29e9c0>> floatingip before_create 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.454794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73e77b29e9c0>> floatingip before_update 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.454967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73e77b29e9c0>> port after_update 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.454967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73e77b29e9c0>> port precommit_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.455183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-28195'] for port_forwarding_plugin (None), after_init {{(pid=92285) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 27 15:57:37.455262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73e77bf35610>> port_forwarding after_create 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.455342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73e77bf35610>> router_interface after_create 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.455437 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73e77bf35610>> port_forwarding after_update 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.455525 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73e77bf35610>> port_forwarding after_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.455598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73e77bf35610>> router_interface after_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.455778 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 27 15:57:37.455882 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension
May 27 15:57:37.456349 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._prevent_segment_delete_with_subnet_associated of <neutron.services.segments.plugin.Plugin object at 0x73e77b29c200>> segment before_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.456464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._validate_auto_address_subnet_delete of <neutron.services.segments.plugin.Plugin object at 0x73e77b29c200>> subnet precommit_delete_associations 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.456812 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_addition_to_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73e77b29e960>> segment_host_mapping after_create 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.456940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_removal_from_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73e77b29e960>> segment_host_mapping after_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.457005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73e77b29e960>> port after_create 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.457074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73e77b29e960>> port after_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.457148 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73e77b29e960>> port after_update 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.457256 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_segment_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73e77b29e960>> segment after_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.457305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_created of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73e77b29e960>> subnet after_create 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.457424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73e77b29e960>> subnet after_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.457508 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73e77b29e960>> subnet after_update 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.540458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x73e77b28e810>> subnet after_create 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.540649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_delete of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x73e77b28e810>> subnet after_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.540649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x73e77b28e810>> subnet before_create 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.540917 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_update of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x73e77b28e810>> subnet before_update 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.543888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_resource_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x73e77bffe870>> security_group after_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.544040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_target_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x73e77bffe870>> port after_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.544202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-7830593'] for logging-plugin (None), after_init {{(pid=92285) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 27 15:57:37.544463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x73e77b28e870>> security_group_rule after_create 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.544576 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x73e77b28e870>> security_group_rule after_update 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:37.544706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x73e77b28e870>> security_group_rule after_delete 55550000, False {{(pid=92285) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 27 15:57:46.969641 np0000001523 neutron-ovn-maintenance-worker[91387]: DEBUG dbcounter [-] [91387] Writing DB stats neutron:SELECT=4 {{(pid=91387) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 15:57:47.545357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-05-27 15:57:47.545130 {{(pid=92285) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}}
May 27 15:57:47.546737 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync port groups started @ 2026-05-27 15:57:47.546496 {{(pid=92285) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}}
May 27 15:57:47.589799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writer thread running {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
May 27 15:57:47.599661 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync port groups completed @ 2026-05-27 15:57:47.599432 {{(pid=92285) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}}
May 27 15:57:47.599777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-05-27 15:57:47.599660 {{(pid=92285) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1152}}
May 27 15:57:47.710488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.services.segments.db [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x73e77be9ffe0>,) {'filters': {'network_id': ['e2510e0f-28d1-484d-8605-6e6fbc1e01e2'], 'is_dynamic': False}} {{(pid=92285) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 27 15:57:47.715438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.services.segments.db [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x73e77be9ffe0>,) {'filters': {'network_id': ['1a1659c0-4943-4985-b3f1-75f0c26af59f'], 'is_dynamic': False}} {{(pid=92285) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 27 15:57:47.719330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync metadata ports started {{(pid=92285) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1097}}
May 27 15:57:47.951570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync metadata ports completed {{(pid=92285) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1149}}
May 27 15:57:47.951693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=92285) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:949}}
May 27 15:57:47.998322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-05-27 15:57:47.998128 {{(pid=92285) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1022}}
May 27 15:57:47.998945 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 27 15:57:47.999179 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=92285) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1027}}
May 27 15:57:47.999260 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=92285) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1086}}
May 27 15:57:47.999362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-05-27 15:57:47.999266 {{(pid=92285) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1321}}
May 27 15:57:47.999473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync ACLs started @ 2026-05-27 15:57:47.999379 {{(pid=92285) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}}
May 27 15:57:48.027326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync ACLs completed @ 2026-05-27 15:57:48.027136 {{(pid=92285) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:431}}
May 27 15:57:48.027433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync routers and router ports started @ 2026-05-27 15:57:48.027324 {{(pid=92285) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:736}}
May 27 15:57:48.181686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 27 15:57:48.181686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync routers and router ports completed @ 2026-05-27 15:57:48.180505 {{(pid=92285) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:944}}
May 27 15:57:48.181686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync port QoS policies started @ 2026-05-27 15:57:48.180669 {{(pid=92285) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1369}}
May 27 15:57:48.181686 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] Starting OVNClientQosExtension
May 27 15:57:48.194832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync port QoS policies completed @ 2026-05-27 15:57:48.194575 {{(pid=92285) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1392}}
May 27 15:57:48.194919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync Floating IP QoS policies started @ 2026-05-27 15:57:48.194776 {{(pid=92285) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1397}}
May 27 15:57:48.194919 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] Starting OVNClientQosExtension
May 27 15:57:48.213657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-05-27 15:57:48.213444 {{(pid=92285) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1420}}
May 27 15:57:48.213767 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync Floating IP NAT rules started @ 2026-05-27 15:57:48.213659 {{(pid=92285) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1425}}
May 27 15:57:48.213962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-05-27 15:57:48.213861 {{(pid=92285) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1451}}
May 27 15:57:48.214053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8cc86ada-fe44-4a34-90ba-ac8fdde9856e None None] OVN-Northbound DB sync process completed @ 2026-05-27 15:57:48.213956 {{(pid=92285) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}}
May 27 15:57:57.546152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-05-27 15:57:57.545847 {{(pid=92285) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1480}}
May 27 15:57:57.546650 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 27 15:57:57.548379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] OVN-SB Sync hostname and physical networks started @ 2026-05-27 15:57:57.548225 {{(pid=92285) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1511}}
May 27 15:57:57.551612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.551855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_virtual_port_parent_hostname {{(pid=92285) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 27 15:57:57.552261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.553107 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started
May 27 15:57:57.553347 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization
May 27 15:57:57.553954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.554082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.554162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.554972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.555260 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.555328 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.555413 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.555498 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.555552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.555824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.555935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.556701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.556772 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.556850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.556915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.557042 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.557817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] Agents populated in the agent cache: {{(pid=92285) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:309}}
May 27 15:57:57.558002 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.558132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] - Agent type: OVN Controller Gateway agent, hostname: np0000001523, name: 9b1006b7-49a8-4735-820f-4e323d9111d3 {{(pid=92285) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:311}}
May 27 15:57:57.558258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.558374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] - Agent type: OVN Neutron agent, hostname: np0000001523, name: 9b1006b7-49a8-4735-820f-4e323d9111d3 {{(pid=92285) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:311}}
May 27 15:57:57.559337 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.559494 np0000001523 neutron-ovn-maintenance-worker[92285]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] New host np0000001523 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron
May 27 15:57:57.559594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.560073 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.560433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.560515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.560600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.560690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' {{(pid=92285) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 27 15:57:57.571469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG oslo_concurrency.lockutils [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=92285) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:511}}
May 27 15:57:57.571711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG oslo_concurrency.lockutils [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.000s {{(pid=92285) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:524}}
May 27 15:57:57.584328 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron_lib.callbacks.manager [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-1181667', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-7999718'] for segment_host_mapping (None), after_create {{(pid=92285) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 27 15:57:57.592334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.services.segments.db [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] Segments {'1d74a028-7065-4269-8c77-c99e0ea79147'} mapped to the host np0000001523 {{(pid=92285) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}}
May 27 15:57:57.596867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG oslo_concurrency.lockutils [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.025s {{(pid=92285) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:543}}
May 27 15:57:57.596867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] OVN-SB Sync hostname and physical networks completed @ 2026-05-27 15:57:57.596687 {{(pid=92285) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1538}}
May 27 15:57:57.608562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:57.608972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_baremetal_ports_dhcp_options {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:57.609059 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Sync baremetal port DHCP options with configuration. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:57.663215 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_baremetal_ports_dhcp_options finished in 0.054 seconds
May 27 15:57:57.663339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:57.663727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_dhcp_options_consistency {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:57.663727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Backfill network_id on DHCP_Options external_ids. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:57.721787 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_dhcp_options_consistency finished in 0.058 seconds
May 27 15:57:57.721941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:57.722080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fair_meter_consistency {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:57.722171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Refresh ACL logging fair meter after configuration reload. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:57.722391 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fair_meter_consistency finished in 0.000 seconds
May 27 15:57:57.722471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:57.722608 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:57.722685 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:57.723740 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:57:57.792782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:57:57.792782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:57.795122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:57:57.795122 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:57.795122 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:57:57.795122 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:57:57.795122 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:57.795122 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:57:57.795122 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:57.795122 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.073 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:57:57.796712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:57.797023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:57:57.797302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:57.797302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:57:57.798943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:57:57.799762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:57.800654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:57:57.800654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:57.800654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:57:57.800654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_for_igmp_snoop_support {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:57.800654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Check for IGMP snoop support. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:57.800654 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_for_igmp_snoop_support finished in 0.000 seconds
May 27 15:57:57.800654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:57.800654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:57.801008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:57.817472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: No inconsistencies found. Skipping {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:437}}
May 27 15:57:57.817660 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_for_inconsistencies finished in 0.018 seconds
May 27 15:57:57.818387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_global_dhcp_opts {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:57.818487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Check global DHCP options consistency. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:57.818707 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_global_dhcp_opts finished in 0.000 seconds
May 27 15:57:57.819016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:57.819178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_localnet_port_has_learn_fdb {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:57.819343 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Ensure localnet ports have learn-fdb set per configuration. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:57.822369 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 27 15:57:57.822780 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_localnet_port_has_learn_fdb finished in 0.003 seconds
May 27 15:57:57.822932 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:57.823128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_network_broadcast_arps_to_all_routers {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:57.824188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Sync broadcast-arps-to-all-routers on external networks. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:57.868251 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_network_broadcast_arps_to_all_routers finished in 0.044 seconds
May 27 15:57:57.868385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:57.868825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_provider_distributed_ports {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:57.868825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Align reside-on-redirect-chassis on VLAN/FLAT distributed ports. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.166342 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_provider_distributed_ports finished in 0.297 seconds
May 27 15:57:58.166496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:58.166678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_redirect_type_router_gateway_ports {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.166807 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Align router gateway redirect-type for provider VLAN/FLAT networks. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.191949 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 27 15:57:58.192254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ddc54787-0210-4433-abf0-04862fa27519 None None] OVN-Southbound DB sync process completed @ 2026-05-27 15:57:58.192009 {{(pid=92285) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1488}}
May 27 15:57:58.238393 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_redirect_type_router_gateway_ports finished in 0.071 seconds
May 27 15:57:58.238503 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:58.238680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.238782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.239844 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:57:58.240045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.240535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:57:58.240535 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.240535 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:57:58.240535 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:57:58.240535 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.240535 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:57:58.240535 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.240535 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.241357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.241821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:57:58.242252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:57:58.242725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.243294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:57:58.243294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:57:58.243294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.243294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:57:58.243294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.243294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:57:58.243294 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: cleanup_old_hash_ring_nodes {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.243294 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Purge stale hash ring nodes older than five days. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.246226 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_hash_ring_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Cleaned up Hash Ring nodes older than 5 days
May 27 15:57:58.246373 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task cleanup_old_hash_ring_nodes finished in 0.004 seconds
May 27 15:57:58.246603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: configure_nb_global {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.246730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply ovn_nb_global settings to NB_Global options. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.246999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:57:58.247411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:57:58.247640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.248069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:57:58.248069 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.248069 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:57:58.248069 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:57:58.248069 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.248069 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:57:58.248069 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.248069 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:57:58.248838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:57:58.249313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.249313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:57:58.249313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.249313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:57:58.249313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:57:58.249313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.249313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:57:58.249313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.249313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task configure_nb_global failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.249712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:57:58.250140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:57:58.250857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:57:58.251634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 27 15:57:58.252284 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:58.252670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.252670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.252670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:57:58.252670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:57:58.252670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.252670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:57:58.252670 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.252670 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:57:58.252670 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.252961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.253393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.254128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.254817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:57:58.255666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:57:58.257046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_network_type_and_physnet {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.257765 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set network type and physnet external_ids on logical switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.260752 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_network_type_and_physnet finished in 0.006 seconds
May 27 15:57:58.261116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:58.261116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_ovn_owned_dns_option {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.261116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply ovn_owned option on DNS records per configuration. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.261286 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_ovn_owned_dns_option finished in 0.000 seconds
May 27 15:57:58.261367 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:58.261446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.261522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.261723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:57:58.262123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:57:58.262290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.262731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:57:58.262731 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.262731 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:57:58.262731 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:57:58.262731 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.262731 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:57:58.262731 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.262731 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:57:58.263338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:57:58.263740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.263740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:57:58.263740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.263740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:57:58.263740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:57:58.263740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.263740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:57:58.263740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.263740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.264008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:57:58.264286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:57:58.264637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.264637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:57:58.264637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.264637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.264759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:57:58.265199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:57:58.265666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.265666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:57:58.265666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.265666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:57:58.265666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_lrouter_ports_ext_ids_name_prefix {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.265666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: ('Check missing prefix router_name', 'in external_ids of LRPs.') {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.265666 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_lrouter_ports_ext_ids_name_prefix finished in 0.000 seconds
May 27 15:57:58.265861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:58.265861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.265929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.266174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:57:58.266473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:57:58.266726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:57:58.266837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.267329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:57:58.267329 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.267329 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:57:58.267329 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:57:58.267329 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.267329 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:57:58.267329 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.267329 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.267984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.268215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:57:58.268462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.268462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:57:58.268870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:57:58.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:57:58.269453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:57:58.269453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:57:58.269453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:57:58.269453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_router_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.269453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Sync router.distributed with distributed floating IP config. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.272695 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_router_distributed_flag finished in 0.003 seconds
May 27 15:57:58.272829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:58.272972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_router_static_routes {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.273076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Mark Neutron-owned static routes in OVN external_ids. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.274003 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_router_static_routes finished in 0.001 seconds
May 27 15:57:58.274133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:58.274277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_security_group_with_address_group {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:57:58.274380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Repair Address_Sets and ACLs for address group rules. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:57:58.284441 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_security_group_with_address_group finished in 0.010 seconds
May 27 15:57:58.284597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:57:58.285725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=92285) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 27 15:58:02.722842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:02.725682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:02.725682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:02.725682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:02.754985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:02.755206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:02.757294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:02.758219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:02.758773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:02.759032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:02.759345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:02.759345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:02.759345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:02.759345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:02.759345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:02.759345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:02.759345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:02.759345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:02.759345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:03.238954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:03.239264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:03.239473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:03.240653 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:03.240900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:03.242911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:03.243829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:03.244033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:03.244250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:03.244576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:03.244576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:03.244576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:03.244576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.244576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:03.244576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:03.244576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.244576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:03.244576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.244576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:03.251249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:03.251910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:03.252080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:03.252288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:03.252982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:03.253190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.256226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:03.256889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:03.257125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.257325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:03.257593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:03.257846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:03.258184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:03.258184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:03.258184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.258184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:03.258184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:03.258184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.258184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:03.258184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.258184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:03.261657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:03.261943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:03.262123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:03.262313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:03.263185 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:03.263185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.266081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:03.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:03.266691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.266904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:03.267210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:03.267541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:03.267926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:03.267926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.267926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:03.267926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:03.267926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.267926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:03.267926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.267926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:03.267926 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:03.269263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:03.269263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:03.269263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:03.269547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:03.269759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:03.269848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:03.271879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:03.272165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:03.272393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:03.272649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:03.273004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:03.273004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:03.273004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:03.273004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:03.273004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:03.273004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:03.273004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:03.273004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:03.273004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:03.273004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:07.723881 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:07.724485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:07.724485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:07.725137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:07.753964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:07.754132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:07.754637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:07.754637 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:07.754637 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:07.754637 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:07.754637 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:07.754637 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:07.754637 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:07.754637 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:07.755638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:07.756024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:07.756268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:07.756268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:07.756798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:07.757258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:07.757827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:07.757827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:07.757827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:08.239439 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:08.239748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:08.239884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:08.240350 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:08.240542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.241036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:08.241036 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.241036 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:08.241036 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:08.241036 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.241036 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:08.241036 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.241036 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:08.241672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.242054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:08.242405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:08.242841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.243341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:08.243341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:08.243341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.243341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:08.243341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.243341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:08.251767 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:08.252007 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:08.252124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:08.252190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:08.252512 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:08.252711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.253122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:08.253122 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.253122 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:08.253122 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:08.253122 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.253122 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:08.253122 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.253122 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:08.253657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:08.254098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:08.254098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:08.254098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.254098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:08.254098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:08.254098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.254098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:08.254098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.254098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.254356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:08.254707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:08.255097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:08.255494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.256017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:08.262031 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:08.262276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:08.262413 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:08.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:08.262885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:08.263096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.263537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:08.263537 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.263537 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:08.263537 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:08.263537 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.263537 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:08.263537 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.263537 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:08.264314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:08.264755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:08.264755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:08.264755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.264755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:08.264755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:08.264755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.264755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:08.264755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.264755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.265253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:08.265557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:08.265978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.265978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:08.265978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.265978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.266415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:08.266945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:08.267459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.267459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:08.267459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.267459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:08.269107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:08.269380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:08.269530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:08.269743 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:08.270309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:08.270511 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:08.270630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.271126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:08.271126 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.271126 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:08.271126 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:08.271126 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.271126 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:08.271126 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.271126 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:08.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.272354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:08.272750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.272750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:08.273105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:08.273536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:08.274075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:08.274075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:08.274075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:12.724427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:12.727088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:12.727088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:12.727088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:12.759063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:12.759212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:12.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:12.761538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:12.761876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:12.762216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:12.762553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:12.762553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:12.762553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:12.762553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:12.762553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:12.762553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:12.762553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:12.762553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:12.762553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:13.239901 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:13.240461 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:13.240461 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:13.241133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:13.241360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.242009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:13.242009 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.242009 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:13.242009 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:13.242009 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.242009 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:13.242009 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.242009 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:13.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.242982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:13.243179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.243179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:13.243519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:13.243842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.244162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:13.244162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.244162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:13.252134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:13.252723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:13.252723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:13.252723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:13.252946 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:13.253101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.253493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:13.253493 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.253493 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:13.253493 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:13.253493 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.253493 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:13.253493 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.253493 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:13.254104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:13.254378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:13.254378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:13.254378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.254378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:13.254378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:13.254378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.254378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:13.254378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.254378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:13.254889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:13.255153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.255153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:13.255153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.255153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:13.255311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:13.255580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:13.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:13.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:13.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:13.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:13.262307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:13.262425 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:13.262504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:13.262613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:13.262993 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:13.263166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.263584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:13.263584 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.263584 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:13.263584 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:13.263584 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.263584 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:13.263584 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.263584 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:13.264129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:13.264383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:13.264383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:13.264383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.264383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:13.264383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:13.264383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.264383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:13.264383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.264383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.264717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:13.264909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:13.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:13.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:13.265685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:13.265974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.265974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:13.265974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.265974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:13.269516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:13.269815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:13.269913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:13.270066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:13.270392 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:13.270634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:13.270734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.271151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:13.271151 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.271151 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:13.271151 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:13.271151 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.271151 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:13.271151 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.271151 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:13.271766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.271979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:13.272169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.272169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:13.272504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:13.273191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:13.273507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:13.273507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:13.273507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:17.725356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:17.726642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:17.726642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:17.727732 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:17.761840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:17.762154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:17.762766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:17.762766 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:17.762766 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:17.762766 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:17.762766 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:17.762766 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:17.762766 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:17.762766 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:17.764056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:17.764395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:17.764764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:17.764764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:17.766017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:17.766441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:17.766927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:17.766927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:17.766927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:18.240299 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:18.241642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:18.241642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:18.241642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:18.241642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:18.244356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:18.245185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:18.245614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:18.246074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:18.246688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:18.246688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:18.246688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:18.246688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.246688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:18.246688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:18.246688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.246688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:18.246688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.246688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:18.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:18.252777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:18.252964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:18.253151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:18.253519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:18.253811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.254252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:18.254252 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.254252 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:18.254252 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:18.254252 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.254252 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:18.254252 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.254252 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:18.255077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:18.255558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:18.255558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:18.255558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.255558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:18.255558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:18.255558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.255558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:18.255558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.255558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.256445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:18.256866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:18.257268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.257268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:18.257268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.257268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:18.258141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:18.258639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:18.259168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.259168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:18.259168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:18.259168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.259168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:18.259168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.259168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:18.262507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:18.263587 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:18.263587 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:18.263587 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:18.263587 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:18.263587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.263959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:18.263959 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.263959 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:18.263959 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:18.263959 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.263959 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:18.263959 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.263959 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:18.265645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:18.266087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:18.266528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.266932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.267826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:18.268167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:18.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:18.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:18.269850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:18.270335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:18.270335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:18.270335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:18.270719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:18.270974 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:18.271122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.271649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:18.271649 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.271649 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:18.271649 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:18.271649 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.271649 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:18.271649 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.271649 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:18.273013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:18.273449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:18.273721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:18.274069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:18.274069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:18.274069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:18.274069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:18.274069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:18.274069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:22.726394 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:22.726987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:22.726987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:22.727422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:22.750943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:22.751134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:22.751577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:22.751577 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:22.751577 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:22.751577 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:22.751577 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:22.751577 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:22.751577 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:22.751577 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:22.752301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:22.752680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:22.753065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:22.753065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:22.753366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:22.753735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:22.754059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:22.754059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:22.754059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:23.240833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:23.242203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:23.242301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:23.242756 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:23.242896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.243295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:23.243295 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.243295 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:23.243295 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:23.243295 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.243295 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:23.243295 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.243295 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:23.243990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.244357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:23.244737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:23.245161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.245693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:23.245693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:23.245693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.245693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:23.245693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.245693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:23.252934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:23.253130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:23.253214 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:23.253307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:23.253762 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:23.253939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.254375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:23.254375 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.254375 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:23.254375 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:23.254375 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.254375 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:23.254375 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.254375 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:23.254968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:23.255391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:23.255391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:23.255391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.255391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:23.255391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:23.255391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.255391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:23.255391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.255391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.255695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:23.256023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:23.256410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:23.256835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.257350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:23.262922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:23.263109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:23.263191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:23.263280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:23.263554 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:23.263747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.264130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:23.264130 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.264130 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:23.264130 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:23.264130 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.264130 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:23.264130 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.264130 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:23.264643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:23.265036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:23.265036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:23.265036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.265036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:23.265036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:23.265036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.265036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:23.265036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.265036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.265284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:23.265477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:23.265750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.265750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:23.265750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.265750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.265856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:23.266145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:23.266453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.266453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:23.266453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.266453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:23.270204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:23.270414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:23.270493 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:23.270617 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:23.270874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:23.271031 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:23.271114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.271448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:23.271448 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.271448 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:23.271448 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:23.271448 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.271448 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:23.271448 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.271448 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:23.272047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.272263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:23.272498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.272498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:23.272663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:23.272932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:23.273218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:23.273218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:23.273218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:27.726721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:27.727122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:27.727257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:27.727933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:27.750444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:27.750642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:27.751047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:27.751047 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:27.751047 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:27.751047 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:27.751047 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:27.751047 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:27.751047 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:27.751047 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.024 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:27.751687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:27.751979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:27.752281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:27.752281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:27.752368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:27.752682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:27.753084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:27.753084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:27.753084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:28.242237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:28.242545 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:28.243472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:28.243472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:28.243585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.244071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:28.244071 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.244071 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:28.244071 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:28.244071 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.244071 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:28.244071 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.244071 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:28.245309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.245679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:28.246079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.246079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:28.246349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:28.246808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.247123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:28.247123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.247123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:28.253201 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:28.253424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:28.253550 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:28.253701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:28.254056 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:28.254220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.254653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:28.254653 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.254653 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:28.254653 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:28.254653 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.254653 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:28.254653 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.254653 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:28.255338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:28.255809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:28.255809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:28.255809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.255809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:28.255809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:28.255809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.255809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:28.255809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.255809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:28.256605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:28.256994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.256994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:28.256994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.256994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:28.257343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:28.257693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:28.257998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.257998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:28.257998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:28.257998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.257998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:28.257998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.257998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:28.263160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:28.263348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:28.263463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:28.263608 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:28.263914 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:28.264088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.264476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:28.264476 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.264476 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:28.264476 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:28.264476 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.264476 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:28.264476 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.264476 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:28.265145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:28.265580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:28.265580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:28.265580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.265580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:28.265580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:28.265580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.265580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:28.265580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.265580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.266063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:28.266378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:28.266802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.266802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:28.266802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.266802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.267146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:28.267458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:28.267935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.267935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:28.267935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.267935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:28.270460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:28.270681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:28.270801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:28.270963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:28.271255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:28.271478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:28.271638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.272021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:28.272021 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.272021 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:28.272021 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:28.272021 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.272021 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:28.272021 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.272021 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:28.272690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.273012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:28.273378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.273378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:28.273712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:28.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:28.274388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:28.274388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:28.274388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:32.727213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:32.727652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:32.727708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:32.728318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:32.754684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:32.754833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:32.755220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:32.755220 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:32.755220 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:32.755220 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:32.755220 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:32.755220 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:32.755220 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:32.755220 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:32.755825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:32.756079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:32.756286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:32.756286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:32.756471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:32.756829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:32.757123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:32.757123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:32.757123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:33.242669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:33.242963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:33.243067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:33.243799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:33.244008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.244536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:33.244536 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.244536 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:33.244536 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:33.244536 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.244536 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:33.244536 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.244536 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:33.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.245562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:33.245812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.245812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:33.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:33.246472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.246776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:33.246776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.246776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:33.253478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:33.253688 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:33.253722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:33.253814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:33.254311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:33.254532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.255026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:33.255026 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.255026 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:33.255026 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:33.255026 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.255026 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:33.255026 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.255026 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:33.255588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:33.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:33.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:33.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:33.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:33.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:33.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.255884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.256191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:33.256398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:33.256643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.256643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:33.256643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.256643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:33.256890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:33.257196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:33.257520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.257520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:33.257520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:33.257520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.257520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:33.257520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.257520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:33.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:33.263580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:33.263674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:33.263780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:33.264053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:33.264245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.264596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:33.264596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.264596 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:33.264596 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:33.264596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.264596 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:33.264596 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.264596 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:33.265096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:33.265354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:33.265354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:33.265354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.265354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:33.265354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:33.265354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.265354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:33.265354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.265354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.265687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:33.265911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:33.266158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.266158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:33.266158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.266158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.266361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:33.266841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:33.267250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.267250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:33.267250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.267250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:33.270775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:33.270979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:33.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:33.271198 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:33.271465 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:33.271667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:33.271758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.272138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:33.272138 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.272138 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:33.272138 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:33.272138 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.272138 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:33.272138 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.272138 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:33.272698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.272977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:33.273349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:33.273777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:33.274239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:33.274239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:33.274239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:33.274239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:33.274239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:33.274239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:37.727754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:37.728130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:37.728252 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:37.728875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:37.754874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:37.755075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:37.755608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:37.755608 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:37.755608 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:37.755608 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:37.755608 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:37.755608 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:37.755608 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:37.755608 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:37.756333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:37.756593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:37.756818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:37.756818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:37.757036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:37.757352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:37.757694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:37.757694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:37.757694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:38.243050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:38.243464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:38.243603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:38.244093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:38.244248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.244673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:38.244673 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.244673 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:38.244673 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:38.244673 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.244673 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:38.244673 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.244673 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:38.245521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.245778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:38.246008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.246008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:38.246550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:38.246877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.247210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:38.247210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.247210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:38.253850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:38.254093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:38.254181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:38.254275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:38.254580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:38.254773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.255189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:38.255189 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.255189 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:38.255189 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:38.255189 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.255189 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:38.255189 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.255189 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:38.255717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:38.256004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:38.256004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:38.256004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.256004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:38.256004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:38.256004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.256004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:38.256004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.256004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.256247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:38.256499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:38.256766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.256766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:38.256766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.256766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:38.256958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:38.257246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:38.257553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.257553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:38.257553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:38.257553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.257553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:38.257553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.257553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:38.263773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:38.263943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:38.264043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:38.264300 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:38.264371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:38.264599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.264993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:38.264993 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.264993 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:38.264993 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:38.264993 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.264993 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:38.264993 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.264993 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:38.265479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:38.265756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:38.265756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:38.265756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.265756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:38.265756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:38.265756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.265756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:38.265756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.265756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.266044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:38.266250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:38.266507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.266507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:38.266507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.266507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.266771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:38.267058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:38.267418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.267418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:38.267418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.267418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:38.271065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:38.271270 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:38.271356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:38.271485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:38.271737 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:38.271933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:38.272041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.272482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:38.272482 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.272482 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:38.272482 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:38.272482 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.272482 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:38.272482 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.272482 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:38.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.273498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:38.273743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.273743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:38.274188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:38.274497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:38.274842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:38.274842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:38.274842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:42.728184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:42.728599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:42.728689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:42.729349 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:42.753456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:42.753456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:42.753891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:42.753891 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:42.753891 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:42.753891 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:42.753891 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:42.753891 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:42.753891 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:42.753891 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:42.754291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:42.754567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:42.754837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:42.754837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:42.754957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:42.755299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:42.755655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:42.755655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:42.755655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:43.243483 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:43.243953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:43.244097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:43.245149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:43.245149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.245339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:43.245339 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.245339 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:43.245339 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:43.245339 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.245339 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:43.245339 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.245339 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:43.246458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.246715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:43.246923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.246923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:43.247592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:43.247877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.248193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:43.248193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.248193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:43.254152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:43.254540 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:43.254697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:43.254843 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:43.255592 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:43.255592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.255765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:43.255765 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.255765 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:43.255765 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:43.255765 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.255765 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:43.255765 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.255765 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:43.256388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:43.256656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:43.256656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:43.256656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.256656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:43.256656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:43.256656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.256656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:43.256656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.256656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.257030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:43.257238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:43.257473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.257473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:43.257473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.257473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:43.257848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:43.258097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:43.258377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.258377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:43.258377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:43.258377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.258377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:43.258377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.258377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:43.263990 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:43.264382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:43.264512 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:43.264676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:43.265547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:43.265547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.265724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:43.265724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.265724 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:43.265724 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:43.265724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.265724 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:43.265724 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.265724 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:43.266408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:43.266681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:43.266681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:43.266681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.266681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:43.266681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:43.266681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.266681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:43.266681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.266681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.267050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:43.267309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:43.267553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.267553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:43.267553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.267553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.267820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:43.268101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:43.268433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.268433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:43.268433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.268433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:43.271391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:43.271605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:43.271716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:43.271852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:43.272150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:43.272394 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:43.272550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.273050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:43.273050 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.273050 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:43.273050 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:43.273050 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.273050 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:43.273050 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.273050 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:43.273660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.273981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:43.274296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.274296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:43.274385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:43.274739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:43.275168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:43.275168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:43.275168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:47.728666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:47.729051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:47.729051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:47.729539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:47.731428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=547,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 15:58:47.754678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:47.754678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:47.754881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:47.754881 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:47.754881 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:47.754881 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:47.754881 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:47.754881 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:47.754881 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:47.754881 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:47.755593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:47.755963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:47.756345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:47.756760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:47.757227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:47.757227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:47.757227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:47.757227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:47.757227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:47.757227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:48.243924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:48.244132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:48.244205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:48.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:48.245316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.245460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:48.245460 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.245460 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:48.245460 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:48.245460 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.245460 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:48.245460 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.245460 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:48.246155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.246376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:48.246592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.246592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:48.246854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:48.247126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.247439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:48.247439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.247439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:48.254422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:48.254570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:48.254668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:48.254761 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:48.255085 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:48.255364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.255833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:48.255833 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.255833 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:48.255833 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:48.255833 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.255833 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:48.255833 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.255833 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:48.256342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:48.256588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:48.256588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:48.256588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.256588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:48.256588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:48.256588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.256588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:48.256588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.256588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.256885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:48.257084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:48.257327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.257327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:48.257327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.257327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:48.257529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:48.257854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:48.258148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.258148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:48.258148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:48.258148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.258148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:48.258148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.258148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:48.264257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:48.264408 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:48.264492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:48.264586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:48.265296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:48.265296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.265423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:48.265423 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.265423 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:48.265423 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:48.265423 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.265423 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:48.265423 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.265423 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:48.265928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:48.266167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:48.266167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:48.266167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.266167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:48.266167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:48.266167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.266167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:48.266167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.266167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.266485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:48.266699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:48.266923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.266923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:48.266923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.266923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.267155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:48.267453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:48.267758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.267758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:48.267758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.267758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:48.271628 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:48.271784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:48.271863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:48.271972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:48.272215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:48.272470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:48.272687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.273118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:48.273118 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.273118 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:48.273118 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:48.273118 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.273118 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:48.273118 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.273118 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:48.273685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.273887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:48.274114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.274114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:48.274299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:48.274563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:48.274866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:48.274866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:48.274866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:52.729042 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:52.729488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:52.729488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:52.730086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:52.761401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:52.761401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:52.761659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:52.761659 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:52.761659 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:52.761659 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:52.761659 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:52.761659 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:52.761659 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:52.761659 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:52.762659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:52.762884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:52.763097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:52.763097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:52.763664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:52.763951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:52.764269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:52.764269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:52.764269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:53.244367 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:53.244646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:53.244783 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:53.245541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:53.245761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.246436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:53.246436 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.246436 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:53.246436 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:53.246436 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.246436 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:53.246436 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.246436 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:53.247461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.247699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:53.247913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.247913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:53.248641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:53.248902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.249247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:53.249247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.249247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:53.254776 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:53.255006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:53.255116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:53.255245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:53.255811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:53.255811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.256315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:53.256315 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.256315 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:53.256315 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:53.256315 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.256315 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:53.256315 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.256315 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:53.257230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:53.257517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:53.257517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:53.257517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.257517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:53.257517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:53.257517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.257517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:53.257517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.257517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.258135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:53.258369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:53.258586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.258586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:53.258586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.258586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:53.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:53.259569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:53.259933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.259933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:53.259933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:53.259933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.259933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:53.259933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.259933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:53.264486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:53.264723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:53.264849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:53.264988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:53.265381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:53.265567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.266040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:53.266040 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.266040 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:53.266040 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:53.266040 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.266040 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:53.266040 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.266040 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:53.266902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:53.267249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:53.267249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:53.267249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.267249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:53.267249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:53.267249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.267249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:53.267249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.267249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.267861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:53.268097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:53.268372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.268372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:53.268372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.268372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.269051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:53.269377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:53.269726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.269726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:53.269726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.269726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:53.271914 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:53.272171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:53.272311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:53.272489 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:53.272961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:53.273178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:53.273287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.273774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:53.273774 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.273774 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:53.273774 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:53.273774 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.273774 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:53.273774 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.273774 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:53.274692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.275039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:53.275301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.275301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:53.275796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:53.276083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:53.276425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:53.276425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:53.276425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:57.729429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:57.730007 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:57.730007 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:57.730477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:57.755383 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:57.755566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:57.755971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:57.755971 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:57.755971 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:57.755971 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:57.755971 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:57.755971 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:57.755971 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:57.755971 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:57.756561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:57.756831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:57.757167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:57.757167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:57.757249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:57.757755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:57.758409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:57.758409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:57.758409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:58.244822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:58.245204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:58.245315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:58.245829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:58.246048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.246537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:58.246537 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.246537 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:58.246537 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:58.246537 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.246537 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:58.246537 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.246537 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:58.247468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.247798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:58.248039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.248039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:58.248249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:58.248553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.248963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:58.248963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.248963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:58.255124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:58.255379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:58.255470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:58.255562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:58:58.255905 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:58.256099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.256561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:58.256561 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.256561 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:58.256561 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:58.256561 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.256561 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:58.256561 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.256561 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:58.257364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:58.257685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:58.257685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:58.257685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.257685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:58.257685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:58.257685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.257685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:58.257685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.257685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.258085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:58.258336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:58.258557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.258557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:58.258557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.258557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:58.258984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:58.259319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:58.259646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.259646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:58.259646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:58.259646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.259646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:58.259646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.259646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:58.264778 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:58.265037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:58.265172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:58.265313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:58.265632 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:58.265840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.266287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:58.266287 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.266287 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:58.266287 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:58.266287 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.266287 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:58.266287 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.266287 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:58:58.266868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:58:58.267186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:58.267186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:58:58.267186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.267186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:58:58.267186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:58:58.267186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.267186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:58:58.267186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.267186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.267439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:58.267680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:58.267966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.267966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:58.267966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.267966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.268165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:58.268555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:58.268931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.268931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:58.268931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.268931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:58:58.272227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:58:58.272461 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:58:58.272558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:58:58.272701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:58:58.273018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:58.273254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:58:58.273344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.273768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:58:58.273768 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.273768 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:58:58.273768 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:58:58.273768 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.273768 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:58:58.273768 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.273768 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:58:58.274393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:58:58.274879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.274879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:58:58.275291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:58:58.275600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:58:58.275973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:58:58.275973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:58:58.275973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:02.729862 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:02.730277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:02.730277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:02.730903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:02.753364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:02.753642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:02.754038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:02.754038 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:02.754038 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:02.754038 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:02.754038 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:02.754038 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:02.754038 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:02.754038 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.024 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:02.754722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:02.754963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:02.755170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:02.755170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:02.755382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:02.755687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:02.756004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:02.756004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:02.756004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:03.245261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:03.245620 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:03.245764 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:03.246299 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:03.246468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.246936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:03.246936 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.246936 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:03.246936 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:03.246936 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.246936 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:03.246936 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.246936 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:03.247635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.248159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:03.248354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:03.248595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.248907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:03.248907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:03.248907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.248907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:03.248907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.248907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:03.255408 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:03.255602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:03.255706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:03.255792 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:03.256077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:03.256237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.256598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:03.256598 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.256598 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:03.256598 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:03.256598 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.256598 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:03.256598 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.256598 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:03.257146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:03.257406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:03.257406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:03.257406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.257406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:03.257406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:03.257406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.257406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:03.257406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.257406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.257819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:03.258068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:03.258292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.258292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:03.258292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.258292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:03.258602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:03.259201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:03.259494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.259494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:03.259494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:03.259494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.259494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:03.259494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.259494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:03.265055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:03.265259 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:03.265345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:03.265446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:03.265702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:03.265862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.266249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:03.266249 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.266249 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:03.266249 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:03.266249 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.266249 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:03.266249 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.266249 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:03.266773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:03.267019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:03.267019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:03.267019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.267019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:03.267019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:03.267019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.267019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:03.267019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.267019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.267362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:03.267990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:03.268271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:03.268478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.268780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:03.272595 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:03.272818 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:03.272907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:03.273017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:03.273275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:03.273426 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:03.273507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.273929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:03.273929 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.273929 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:03.273929 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:03.273929 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.273929 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:03.273929 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.273929 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:03.274506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.274736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:03.274922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.274922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:03.275233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:03.275487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:03.275788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:03.275788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:03.275788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:07.730334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:07.730769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:07.730769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:07.731973 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:07.757574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:07.758018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:07.758748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:07.758748 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:07.758748 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:07.758748 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:07.758748 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:07.758748 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:07.758748 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:07.758748 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:07.760660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:07.761142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:07.761641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:07.762162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:07.762476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:07.762476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:07.762476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:07.762476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:07.762476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:07.762476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:08.245852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:08.246581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:08.246581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:08.247658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:08.247658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:08.250183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:08.250728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:08.251140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:08.251542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:08.252113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:08.252113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:08.252113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:08.252113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.252113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:08.252113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:08.252113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.252113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:08.252113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.252113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:08.255788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:08.256399 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:08.256399 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:08.256399 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:08.257244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:08.257244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.260491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:08.261210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:08.261649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.262017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:08.262395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:08.262839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:08.263385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:08.263385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:08.263385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.263385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:08.263385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:08.263385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.263385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:08.263385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.263385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:08.265575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:08.266348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:08.266348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:08.266348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:08.267487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:08.267487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.267770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:08.267770 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.267770 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:08.267770 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:08.267770 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.267770 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:08.267770 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.267770 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:08.270649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:08.271235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:08.271806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.272293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.273368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:08.274063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:08.274803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.274803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:08.274803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.274803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:08.275198 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:08.275509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:08.276156 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:08.276156 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:08.277079 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:08.277079 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:08.277079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:08.279563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:08.280101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:08.280507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:08.280937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:08.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:08.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:08.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:08.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:08.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:08.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:08.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:08.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:08.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:08.281541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:12.730727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:12.731268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:12.731268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:12.731874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:12.757584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:12.757831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:12.758368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:12.758368 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:12.758368 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:12.758368 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:12.758368 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:12.758368 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:12.758368 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:12.758368 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:12.759055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:12.759293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:12.759570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:12.759570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:12.759758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:12.760268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:12.760763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:12.760763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:12.760763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:13.246321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:13.246837 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:13.247036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:13.247742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:13.248010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.248637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:13.248637 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.248637 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:13.248637 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:13.248637 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.248637 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:13.248637 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.248637 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:13.249798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.250106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:13.250314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.250314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:13.251065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:13.251347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.251688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:13.251688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.251688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:13.256088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:13.256471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:13.256615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:13.256800 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:13.257163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:13.257338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.257857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:13.257857 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.257857 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:13.257857 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:13.257857 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.257857 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:13.257857 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.257857 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:13.258923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:13.259242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:13.259242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:13.259242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.259242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:13.259242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:13.259242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.259242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:13.259242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.259242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.259883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:13.260202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:13.260585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.260585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:13.260585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.260585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:13.260768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:13.261250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:13.261807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.261807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:13.261807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:13.261807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.261807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:13.261807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.261807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:13.265834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:13.266151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:13.266271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:13.266392 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:13.266714 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:13.266921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.267402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:13.267402 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.267402 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:13.267402 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:13.267402 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.267402 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:13.267402 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.267402 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:13.268441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:13.268719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:13.268719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:13.268719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.268719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:13.268719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:13.268719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.268719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:13.268719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.268719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.269552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:13.269830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:13.270066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.270066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:13.270066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.270066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.270933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:13.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:13.271713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.271713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:13.271713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.271713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:13.275529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:13.276016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:13.276208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:13.276478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:13.276821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:13.277045 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:13.277138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.277584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:13.277584 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.277584 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:13.277584 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:13.277584 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.277584 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:13.277584 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.277584 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:13.278478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.278736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:13.278994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.278994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:13.279276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:13.279588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:13.280050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:13.280050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:13.280050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:17.731150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:17.731544 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:17.731544 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:17.732080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:17.756362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:17.756526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:17.756950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:17.756950 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:17.756950 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:17.756950 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:17.756950 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:17.756950 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:17.756950 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:17.756950 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:17.757534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:17.757769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:17.757974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:17.757974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:17.758271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:17.758524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:17.758848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:17.758848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:17.758848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:18.246789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:18.247020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:18.247105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:18.247505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:18.247642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.248061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:18.248061 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.248061 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:18.248061 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:18.248061 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.248061 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:18.248061 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.248061 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:18.248678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.248923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:18.249126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.249126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:18.249353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:18.249599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.249916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:18.249916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.249916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:18.256372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:18.256564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:18.256679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:18.256787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:18.257015 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:18.257145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.257448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:18.257448 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.257448 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:18.257448 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:18.257448 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.257448 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:18.257448 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.257448 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:18.257963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:18.258216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:18.258216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:18.258216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.258216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:18.258216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:18.258216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.258216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:18.258216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.258216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.258483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:18.258694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:18.258904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.258904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:18.258904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.258904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:18.259187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:18.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:18.260203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.260203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:18.260203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:18.260203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.260203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:18.260203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.260203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:18.266080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:18.266232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:18.266318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:18.266402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:18.266597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:18.266785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.267091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:18.267091 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.267091 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:18.267091 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:18.267091 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.267091 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:18.267091 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.267091 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:18.267549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:18.267819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:18.267819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:18.267819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.267819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:18.267819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:18.267819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.267819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:18.267819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.267819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.268127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:18.268322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:18.268522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.268522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:18.268522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.268522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.268821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:18.269079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:18.269359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.269359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:18.269359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.269359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:18.275800 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:18.275933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:18.276008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:18.276137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:18.276314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:18.276456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:18.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.276920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:18.276920 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.276920 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:18.276920 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:18.276920 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.276920 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:18.276920 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.276920 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:18.277411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.277617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:18.277854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.277854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:18.278054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:18.278286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:18.278558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:18.278558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:18.278558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:22.731615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:22.732054 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:22.732169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:22.732851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:22.757405 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:22.757706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:22.758333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:22.758333 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:22.758333 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:22.758333 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:22.758333 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:22.758333 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:22.758333 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:22.758333 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:22.759024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:22.759477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:22.759727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:22.759995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:22.760330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:22.760330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:22.760330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:22.760330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:22.760330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:22.760330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:23.247167 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:23.247422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:23.247530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:23.247964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:23.248100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.248491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:23.248491 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.248491 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:23.248491 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:23.248491 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.248491 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:23.248491 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.248491 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:23.249150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.249531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:23.249816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.249816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:23.249969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:23.250583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.251354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:23.251354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.251354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:23.256589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:23.256802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:23.256888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:23.256996 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:23.257245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:23.257378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.257724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:23.257724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.257724 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:23.257724 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:23.257724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.257724 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:23.257724 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.257724 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:23.258207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:23.258509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:23.258509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:23.258509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.258509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:23.258509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:23.258509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.258509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:23.258509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.258509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.258801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:23.259071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:23.259360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.259360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:23.259360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.259360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:23.259489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:23.260123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:23.260801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.260801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:23.260801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:23.260801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.260801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:23.260801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.260801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:23.266281 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:23.266440 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:23.266522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:23.266639 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:23.266866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:23.267053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.267365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:23.267365 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.267365 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:23.267365 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:23.267365 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.267365 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:23.267365 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.267365 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:23.267929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:23.268197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:23.268197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:23.268197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.268197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:23.268197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:23.268197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.268197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:23.268197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.268197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.268711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:23.268936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:23.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:23.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.269160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.269646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:23.269980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:23.270302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.270302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:23.270302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.270302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:23.275989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:23.276158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:23.276266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:23.276406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:23.276599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:23.276794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:23.276876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.277225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:23.277225 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.277225 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:23.277225 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:23.277225 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.277225 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:23.277225 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.277225 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:23.277879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.278188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:23.278387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.278387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:23.278762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:23.279116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:23.279454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:23.279454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:23.279454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:27.732191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:27.732671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:27.732671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:27.733418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:27.761203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:27.761412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:27.762125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:27.762125 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:27.762125 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:27.762125 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:27.762125 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:27.762125 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:27.762125 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:27.762125 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:27.763661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:27.763997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:27.764206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:27.764469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:27.764785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:27.764785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:27.764785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:27.764785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:27.764785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:27.764785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:28.247641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:28.248073 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:28.248240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:28.248870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:28.249084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.249546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:28.249546 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.249546 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:28.249546 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:28.249546 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.249546 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:28.249546 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.249546 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:28.250572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.250839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:28.251031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.251031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:28.251748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:28.252016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.252296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:28.252296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.252296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:28.256896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:28.257269 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:28.257410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:28.257566 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:28.257898 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:28.258049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.258495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:28.258495 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.258495 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:28.258495 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:28.258495 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.258495 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:28.258495 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.258495 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:28.259401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:28.259758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:28.259758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:28.259758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.259758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:28.259758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:28.259758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.259758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:28.259758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.259758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.260232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:28.260584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:28.260891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.260891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:28.260891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.260891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:28.261258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:28.261670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:28.261927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.261927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:28.261927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:28.261927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.261927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:28.261927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.261927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:28.266605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:28.266969 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:28.267134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:28.267307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:28.267667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:28.267866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.268294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:28.268294 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.268294 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:28.268294 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:28.268294 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.268294 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:28.268294 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.268294 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:28.269243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:28.269503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:28.269503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:28.269503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.269503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:28.269503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:28.269503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.269503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:28.269503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.269503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.270216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:28.270412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:28.270655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.270655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:28.270655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.270655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.271418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:28.271762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:28.272046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.272046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:28.272046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.272046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:28.276261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:28.276702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:28.276863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:28.277083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:28.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:28.277557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:28.277656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.278102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:28.278102 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.278102 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:28.278102 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:28.278102 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.278102 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:28.278102 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.278102 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:28.279147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.279370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:28.279584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.279584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:28.280066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:28.280328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:28.280654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:28.280654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:28.280654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:32.732691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:32.733063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:32.733094 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:32.733716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:32.759457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:32.760365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:32.760365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:32.760365 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:32.760365 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:32.760365 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:32.760365 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:32.760365 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:32.760365 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:32.760365 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:32.760966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:32.761392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:32.761850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:32.761850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:32.762052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:32.762389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:32.762759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:32.762759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:32.762759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:33.248021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:33.248289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:33.248390 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:33.248880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:33.249037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.249504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:33.249504 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.249504 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:33.249504 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:33.249504 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.249504 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:33.249504 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.249504 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:33.250129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.250365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:33.250599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.250599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:33.250856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:33.251235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.251637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:33.251637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.251637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:33.257206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:33.257379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:33.257473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:33.257557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:33.257835 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:33.257977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.258282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:33.258282 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.258282 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:33.258282 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:33.258282 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.258282 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:33.258282 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.258282 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:33.258806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:33.259092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:33.259092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:33.259092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.259092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:33.259092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:33.259092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.259092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:33.259092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.259092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.259326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:33.259579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:33.259875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.259875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:33.259875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.259875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:33.260005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:33.260289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:33.260648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.260648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:33.260648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:33.260648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.260648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:33.260648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.260648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:33.267098 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:33.267375 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:33.267468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:33.267567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:33.268466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:33.268466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.268764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:33.268764 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.268764 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:33.268764 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:33.268764 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.268764 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:33.268764 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.268764 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:33.269266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:33.269735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:33.269735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:33.269735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.269735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:33.269735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:33.269735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.269735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:33.269735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.269735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.270006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:33.270336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:33.270750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:33.271156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.271707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:33.276682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:33.276912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:33.277004 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:33.277137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:33.277424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:33.277615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:33.277747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.278248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:33.278248 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.278248 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:33.278248 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:33.278248 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.278248 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:33.278248 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.278248 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:33.278849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.279202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:33.279576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:33.280002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:33.280614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:33.280614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:33.280614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:33.280614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:33.280614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:33.280614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:37.733274 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:37.733709 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:37.733709 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:37.734207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:37.758020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:37.758155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:37.758547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:37.758547 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:37.758547 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:37.758547 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:37.758547 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:37.758547 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:37.758547 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:37.758547 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:37.759166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:37.759434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:37.759713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:37.759713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:37.759861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:37.760194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:37.760530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:37.760530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:37.760530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:38.248451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:38.248760 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:38.248875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:38.249364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:38.249503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.249928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:38.249928 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.249928 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:38.249928 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:38.249928 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.249928 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:38.249928 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.249928 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:38.250639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.250975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:38.251388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:38.251791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.252349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:38.252349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:38.252349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.252349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:38.252349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.252349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:38.257497 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:38.257710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:38.257789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:38.257885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:38.258126 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:38.258289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.258660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:38.258660 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.258660 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:38.258660 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:38.258660 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.258660 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:38.258660 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.258660 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:38.259121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:38.259554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:38.259554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:38.259554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.259554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:38.259554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:38.259554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.259554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:38.259554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.259554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.259792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:38.260097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:38.260457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:38.260861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.261320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:38.267469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:38.267707 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:38.267803 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:38.267880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:38.268152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:38.268337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.268725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:38.268725 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.268725 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:38.268725 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:38.268725 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.268725 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:38.268725 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.268725 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:38.269189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:38.269595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:38.269595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:38.269595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.269595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:38.269595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:38.269595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.269595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:38.269595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.269595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.269840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:38.270168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:38.270593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:38.270999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.271481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:38.276962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:38.277135 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:38.277227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:38.277427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:38.277601 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:38.277779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:38.277869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.278210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:38.278210 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.278210 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:38.278210 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:38.278210 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.278210 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:38.278210 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.278210 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:38.278753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.279062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:38.279405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:38.279808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:38.280301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:38.280301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:38.280301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:38.280301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:38.280301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:38.280301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:42.733698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:42.736088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:42.736088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:42.736088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:42.763234 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:42.763569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:42.764114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:42.764114 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:42.764114 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:42.764114 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:42.764114 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:42.764114 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:42.764114 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:42.764114 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:42.765119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:42.765484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:42.765818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:42.765818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:42.766814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:42.767253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:42.767773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:42.767773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:42.767773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:43.249749 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:43.249749 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:43.249749 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:43.250968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:43.250968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.251218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:43.251218 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.251218 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:43.251218 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:43.251218 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.251218 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:43.251218 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.251218 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:43.252485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.252958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:43.253362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.253362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:43.254665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:43.255220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.255848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:43.255848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.255848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:43.258402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:43.258402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:43.258402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:43.258402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:43.258896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:43.259169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.259725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:43.259725 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.259725 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:43.259725 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:43.259725 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.259725 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:43.259725 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.259725 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:43.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:43.263543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:43.264049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:43.264911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:43.265431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:43.266090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.266090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:43.266090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:43.266090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.266090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:43.266090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.266090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:43.267761 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:43.268112 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:43.268341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:43.268585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:43.268986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:43.269352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.269975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:43.269975 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.269975 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:43.269975 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:43.269975 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.269975 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:43.269975 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.269975 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:43.271157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:43.271780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:43.271780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:43.271780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.271780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:43.271780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:43.271780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.271780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:43.271780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.271780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.273074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:43.273501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:43.273956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.273956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:43.273956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.273956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.275574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:43.276245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:43.276891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.276891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:43.276891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.276891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:43.277242 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:43.278972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:43.278972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:43.278972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:43.278972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:43.278972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:43.278972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:43.281396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:43.281931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:43.282394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:43.282894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:43.283552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:43.283552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:43.283552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:43.283552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:43.283552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:43.283552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:43.283552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:43.283552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:43.283552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:43.283552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:47.734731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:47.735154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:47.735289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:47.735950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:47.761582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:47.761767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:47.762265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:47.762265 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:47.762265 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:47.762265 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:47.762265 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:47.762265 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:47.762265 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:47.762265 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:47.763038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:47.763332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:47.763523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:47.763523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:47.763894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:47.764189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:47.764481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:47.764481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:47.764481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:48.249307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:48.249717 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:48.249842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:48.250451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:48.250705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.251189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:48.251189 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.251189 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:48.251189 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:48.251189 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.251189 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:48.251189 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.251189 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:48.251931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.252180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:48.252382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.252382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:48.252669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:48.252942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.253238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:48.253238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.253238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:48.258114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:48.258325 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:48.258408 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:48.258508 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:48.258879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:48.259068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.259471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:48.259471 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.259471 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:48.259471 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:48.259471 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.259471 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:48.259471 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.259471 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:48.260048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:48.260295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:48.260295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:48.260295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.260295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:48.260295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:48.260295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.260295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:48.260295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.260295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.260612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:48.261037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:48.261432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:48.261667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.261936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:48.268133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:48.268344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:48.268423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:48.268529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:48.268941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:48.269170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.269601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:48.269601 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.269601 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:48.269601 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:48.269601 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.269601 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:48.269601 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.269601 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:48.270125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:48.270365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:48.270365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:48.270365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.270365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:48.270365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:48.270365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.270365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:48.270365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.270365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.270692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:48.271095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:48.271488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:48.271721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.272013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:48.277522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:48.277752 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:48.277833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:48.277961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:48.278260 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:48.278445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:48.278534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.278902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:48.278902 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.278902 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:48.278902 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:48.278902 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.278902 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:48.278902 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.278902 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:48.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.279655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:48.279880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.279880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:48.280109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:48.280375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:48.280663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:48.280663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:48.280663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:52.735187 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:52.735886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:52.735886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:52.736396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:52.737828 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=260 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 15:59:52.761236 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:52.761510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:52.762103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:52.762103 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:52.762103 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:52.762103 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:52.762103 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:52.762103 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:52.762103 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:52.762103 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:52.762897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:52.763192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:52.763428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:52.763428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:52.763840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:52.764147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:52.764497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:52.764497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:52.764497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:53.249785 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:53.250055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:53.250166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:53.250756 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:53.251000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.251540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:53.251540 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.251540 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:53.251540 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:53.251540 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.251540 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:53.251540 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.251540 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:53.252286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.252556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:53.252896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.252896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:53.253039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:53.253416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.253794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:53.253794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.253794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:53.258381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:53.258571 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:53.258659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:53.258756 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:53.259129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:53.259340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.259876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:53.259876 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.259876 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:53.259876 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:53.259876 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.259876 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:53.259876 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.259876 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:53.260436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:53.260780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:53.260780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:53.260780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.260780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:53.260780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:53.260780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.260780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:53.260780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.260780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.261076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:53.261309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:53.261545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.261545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:53.261545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.261545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:53.261748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:53.262090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:53.262383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.262383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:53.262383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:53.262383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.262383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:53.262383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.262383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:53.268391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:53.268599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:53.268702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:53.268819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:53.269154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:53.269346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.269781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:53.269781 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.269781 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:53.269781 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:53.269781 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.269781 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:53.269781 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.269781 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:53.270373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:53.270790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:53.270790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:53.270790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.270790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:53.270790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:53.270790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.270790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:53.270790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.270790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.271084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:53.271360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:53.271596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.271596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:53.271596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.271596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.271824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:53.272162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:53.272455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.272455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:53.272455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.272455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:53.277774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:53.277964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:53.278052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:53.278195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:53.278469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:53.278633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:53.278738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.279134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:53.279134 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.279134 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:53.279134 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:53.279134 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.279134 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:53.279134 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.279134 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:53.279858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.280244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:53.280655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.280655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:53.280864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:53.281179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:53.281495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:53.281495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:53.281495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:57.735619 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:57.736092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:57.736092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:57.736672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:57.761599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:57.761814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:57.762339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:57.762339 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:57.762339 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:57.762339 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:57.762339 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:57.762339 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:57.762339 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:57.762339 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:57.763077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:57.763327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:57.763528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:57.763528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:57.763854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:57.764365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:57.764762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:57.764762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:57.764762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:58.250163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:58.250389 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:58.250491 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:58.251089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:58.251273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.251734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:58.251734 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.251734 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:58.251734 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:58.251734 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.251734 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:58.251734 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.251734 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:58.252420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.252640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:58.252883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.252883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:58.253100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:58.253346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.253647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:58.253647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.253647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:58.258634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:58.258817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:58.258891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:58.258989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 15:59:58.259239 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:58.259382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.259741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:58.259741 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.259741 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:58.259741 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:58.259741 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.259741 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:58.259741 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.259741 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:58.260226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:58.260463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:58.260463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:58.260463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.260463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:58.260463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:58.260463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.260463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:58.260463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.260463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.260771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:58.261142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:58.261465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:58.261710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:58.268671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:58.268840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:58.268914 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:58.269008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:58.269287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:58.269444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.269804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:58.269804 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.269804 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:58.269804 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:58.269804 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.269804 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:58.269804 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.269804 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 15:59:58.270287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 15:59:58.270530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:58.270530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 15:59:58.270530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.270530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 15:59:58.270530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 15:59:58.270530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.270530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 15:59:58.270530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.270530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.270871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:58.271063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:58.271279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.271279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:58.271279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.271279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.271578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:58.272175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:58.272575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.272575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:58.272575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.272575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 15:59:58.277999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 15:59:58.278178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 15:59:58.278279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 15:59:58.278419 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 15:59:58.278779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:58.278950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 15:59:58.279037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.279452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 15:59:58.279452 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.279452 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 15:59:58.279452 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 15:59:58.279452 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.279452 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 15:59:58.279452 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.279452 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 15:59:58.280046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.280239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 15:59:58.280439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.280439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 15:59:58.280781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 15:59:58.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 15:59:58.281867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 15:59:58.281867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 15:59:58.281867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:02.736121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:02.736619 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:02.736619 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:02.737207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:02.762131 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:02.762387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:02.763006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:02.763006 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:02.763006 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:02.763006 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:02.763006 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:02.763006 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:02.763006 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:02.763006 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:02.763747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:02.764019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:02.764232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:02.764232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:02.764669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:02.764972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:02.765278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:02.765278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:02.765278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:03.250565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:03.250902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:03.251042 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:03.251686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:03.251876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.252353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:03.252353 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.252353 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:03.252353 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:03.252353 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.252353 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:03.252353 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.252353 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:03.253477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.253739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:03.253940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.253940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:03.254683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:03.254958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.255273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:03.255273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.255273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:03.258869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:03.259096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:03.259218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:03.259358 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:03.259666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:03.259864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.260297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:03.260297 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.260297 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:03.260297 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:03.260297 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.260297 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:03.260297 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.260297 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:03.261137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:03.261398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:03.261398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:03.261398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.261398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:03.261398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:03.261398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.261398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:03.261398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.261398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.262049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:03.262286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:03.262500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.262500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:03.262500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.262500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:03.263179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:03.263440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:03.263749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.263749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:03.263749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:03.263749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.263749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:03.263749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.263749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:03.268910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:03.269127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:03.269258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:03.269403 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:03.269694 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:03.269858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.270256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:03.270256 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.270256 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:03.270256 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:03.270256 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.270256 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:03.270256 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.270256 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:03.271108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:03.271396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:03.271396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:03.271396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.271396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:03.271396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:03.271396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.271396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:03.271396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.271396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.272073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:03.272296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:03.272506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.272506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:03.272506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.272506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.273243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:03.273529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:03.273857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.273857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:03.273857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.273857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:03.278211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:03.278412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:03.278556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:03.278737 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:03.278997 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:03.279181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:03.279284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.279774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:03.279774 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.279774 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:03.279774 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:03.279774 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.279774 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:03.279774 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.279774 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:03.280667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.280889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:03.281100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.281100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:03.281311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:03.281881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:03.282375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:03.282375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:03.282375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:07.736564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:07.737006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:07.737006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:07.737643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:07.766563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:07.766803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:07.767286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:07.767286 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:07.767286 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:07.767286 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:07.767286 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:07.767286 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:07.767286 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:07.767286 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:07.768074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:07.768290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:07.768517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:07.768517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:07.768819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:07.769086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:07.769365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:07.769365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:07.769365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:08.251930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:08.252140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:08.252261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:08.252986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:08.253227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.253850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:08.253850 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.253850 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:08.253850 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:08.253850 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.253850 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:08.253850 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.253850 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:08.254548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.254965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:08.255355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:08.255877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.256502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:08.256502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:08.256502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.256502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:08.256502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.256502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:08.259193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:08.259428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:08.259541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:08.259678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:08.260094 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:08.260311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.260853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:08.260853 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.260853 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:08.260853 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:08.260853 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.260853 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:08.260853 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.260853 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:08.261361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:08.261819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:08.261819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:08.261819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.261819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:08.261819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:08.261819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.261819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:08.261819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.261819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.262085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:08.262449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:08.262858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:08.263262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.263834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:08.269169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:08.269340 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:08.269431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:08.269571 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:08.269955 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:08.270182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.270781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:08.270781 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.270781 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:08.270781 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:08.270781 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.270781 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:08.270781 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.270781 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:08.271348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:08.271791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:08.271791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:08.271791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.271791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:08.271791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:08.271791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.271791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:08.271791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.271791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.272055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:08.272439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:08.272876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:08.273293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.273902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:08.278472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:08.278713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:08.278773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:08.278907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:08.279360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:08.279640 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:08.279759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.280277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:08.280277 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.280277 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:08.280277 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:08.280277 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.280277 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:08.280277 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.280277 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:08.280846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.281215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:08.281595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:08.282068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:08.282666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:08.282666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:08.282666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:08.282666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:08.282666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:08.282666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:12.736963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:12.737482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:12.737482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:12.738013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:12.764772 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:12.764965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:12.765453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:12.765453 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:12.765453 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:12.765453 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:12.765453 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:12.765453 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:12.765453 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:12.765453 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:12.766219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:12.766512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:12.766762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:12.766762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:12.767028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:12.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:12.767640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:12.767640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:12.767640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:13.252236 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:13.252605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:13.252731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:13.253225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:13.253399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.253859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:13.253859 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.253859 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:13.253859 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:13.253859 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.253859 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:13.253859 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.253859 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:13.254594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.254842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:13.255055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.255055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:13.255343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:13.255618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.255973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:13.255973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.255973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:13.259466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:13.259676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:13.259767 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:13.259856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:13.260120 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:13.260281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.260671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:13.260671 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.260671 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:13.260671 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:13.260671 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.260671 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:13.260671 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.260671 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:13.261192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:13.261477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:13.261477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:13.261477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.261477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:13.261477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:13.261477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.261477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:13.261477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.261477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.261845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:13.262162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:13.262437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:13.262684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.262975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:13.269400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:13.269592 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:13.269699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:13.269791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:13.270041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:13.270202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.270581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:13.270581 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.270581 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:13.270581 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:13.270581 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.270581 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:13.270581 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.270581 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:13.271117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:13.271362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:13.271362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:13.271362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.271362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:13.271362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:13.271362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.271362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:13.271362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.271362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.271688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:13.271927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:13.272153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.272153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:13.272153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.272153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:13.273078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:13.273374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.273374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:13.273374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.273374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:13.278782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:13.278980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:13.279055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:13.279183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:13.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:13.279591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:13.279693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.280070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:13.280070 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.280070 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:13.280070 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:13.280070 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.280070 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:13.280070 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.280070 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:13.280705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.280930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:13.281128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.281128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:13.281365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:13.281652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:13.281954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:13.281954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:13.281954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:17.737376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:17.737870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:17.737870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:17.738262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:17.763153 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:17.763367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:17.763917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:17.763917 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:17.763917 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:17.763917 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:17.763917 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:17.763917 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:17.763917 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:17.763917 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:17.764664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:17.764894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:17.765105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:17.765105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:17.765373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:17.765643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:17.765932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:17.765932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:17.765932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:18.252780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:18.253142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:18.253298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:18.253991 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:18.254213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.254852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:18.254852 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.254852 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:18.254852 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:18.254852 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.254852 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:18.254852 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.254852 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:18.255738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.255951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:18.256232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.256232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:18.256508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:18.256814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.257114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:18.257114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.257114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:18.259803 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:18.260197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:18.260297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:18.260394 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:18.260810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:18.260974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.261462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:18.261462 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.261462 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:18.261462 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:18.261462 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.261462 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:18.261462 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.261462 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:18.262351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:18.262942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:18.263405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.263803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:18.264161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:18.264698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:18.265268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.265268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:18.265268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:18.265268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.265268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:18.265268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.265268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:18.269749 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:18.270076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:18.270204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:18.270355 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:18.270720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:18.270990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.271529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:18.271529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.271529 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:18.271529 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:18.271529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.271529 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:18.271529 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.271529 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:18.272279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:18.272599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:18.272599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:18.272599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.272599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:18.272599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:18.272599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.272599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:18.272599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.272599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.273157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:18.273428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:18.273757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.273757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:18.273757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.273757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.274201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:18.274519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:18.274871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.274871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:18.274871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.274871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:18.279092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:18.279287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:18.279392 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:18.279552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:18.279846 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:18.280084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:18.280192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.280612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:18.280612 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.280612 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:18.280612 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:18.280612 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.280612 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:18.280612 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.280612 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:18.281417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.281811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:18.282211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.282211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:18.282382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:18.282888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:18.283432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:18.283432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:18.283432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:22.737873 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:22.738539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:22.738539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:22.738800 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:22.770172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:22.770172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:22.770397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:22.770397 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:22.770397 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:22.770397 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:22.770397 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:22.770397 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:22.770397 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:22.770397 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:22.771249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:22.771520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:22.771778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:22.771778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:22.771987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:22.772274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:22.772599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:22.772599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:22.772599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:23.253232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:23.253652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:23.253652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:23.255085 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:23.255322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.255997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:23.255997 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.255997 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:23.255997 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:23.255997 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.255997 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:23.255997 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.255997 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:23.256749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.257000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:23.257232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.257232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:23.257679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:23.257983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.258308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:23.258308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.258308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:23.260247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:23.260659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:23.260659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:23.260659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:23.261109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:23.261349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.261853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:23.261853 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.261853 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:23.261853 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:23.261853 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.261853 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:23.261853 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.261853 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:23.262542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:23.262920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:23.262920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:23.262920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.262920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:23.262920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:23.262920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.262920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:23.262920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.262920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.263397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:23.263643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:23.263871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.263871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:23.263871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.263871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:23.264377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:23.264682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:23.265121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.265121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:23.265121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:23.265121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.265121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:23.265121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.265121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:23.270115 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:23.270296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:23.270402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:23.270549 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:23.271066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:23.271374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.272020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:23.272020 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.272020 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:23.272020 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:23.272020 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.272020 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:23.272020 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.272020 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:23.272804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:23.273446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:23.273446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:23.273446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.273446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:23.273446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:23.273446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.273446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:23.273446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.273446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.273707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:23.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:23.274412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.274412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:23.274412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.274412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.274929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:23.275437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:23.275784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.275784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:23.275784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.275784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:23.279443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:23.279707 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:23.279788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:23.280015 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:23.280413 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:23.280644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:23.280760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.281259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:23.281259 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.281259 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:23.281259 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:23.281259 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.281259 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:23.281259 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.281259 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:23.282101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.282485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:23.282932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.282932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:23.283161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:23.283710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:23.284292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:23.284292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:23.284292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:27.738190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:27.739347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:27.739347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:27.739347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:27.775003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:27.775249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:27.775968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:27.775968 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:27.775968 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:27.775968 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:27.775968 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:27.775968 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:27.775968 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:27.775968 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:27.776784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:27.777099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:27.777317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:27.777317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:27.777616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:27.778129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:27.778587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:27.778587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:27.778587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:28.253673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:28.254076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:28.254228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:28.255916 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.256118 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.256287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.257110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.257298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.257450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.257769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.257769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.257890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-c854b062-68ff-4ab5-87d6-62227b915b10, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.258690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.258690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:28.258690 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.258690 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:28.258690 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:28.258690 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.258690 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:28.258690 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.258690 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:28.259377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.259843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:28.260234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.260234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:28.260476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:28.260819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.261246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:28.261246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.261246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:28.261246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:28.261246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:28.261433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:28.261479 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:28.261756 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.261890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.262230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:28.262230 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.262230 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:28.262230 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:28.262230 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.262230 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:28.262230 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.262230 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:28.262907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:28.263382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:28.263382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:28.263382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.263382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:28.263382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:28.263382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.263382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:28.263382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.263382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.263793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:28.264187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:28.264649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.264649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:28.264649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.264649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:28.264907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:28.265261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:28.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:28.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:28.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:28.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:28.270503 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:28.270868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:28.270984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:28.271097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:28.271393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.271602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.272052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:28.272052 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.272052 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:28.272052 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:28.272052 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.272052 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:28.272052 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.272052 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:28.272681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:28.273095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:28.273095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:28.273095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.273095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:28.273095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:28.273095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.273095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:28.273095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.273095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.273549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:28.273935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:28.274361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.274361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:28.274361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.274361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.274684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:28.275045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:28.275576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.275576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:28.275576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.275576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:28.279869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:28.280187 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:28.280286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:28.280451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:28.280794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.280999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:28.281206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.281668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:28.281668 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.281668 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:28.281668 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:28.281668 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.281668 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:28.281668 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.281668 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:28.282256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.282649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:28.282985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:28.283283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:28.283581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:28.283581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:28.283581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:28.283581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:28.283581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:28.283581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:32.738718 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:32.739162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:32.739162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:32.739791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:32.769758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:32.769918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:32.770478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:32.770478 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:32.770478 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:32.770478 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:32.770478 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:32.770478 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:32.770478 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:32.770478 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:32.771221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:32.771596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:32.771977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:32.772397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:32.772936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:32.772936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:32.772936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:32.772936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:32.772936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:32.772936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:33.255159 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:33.255159 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:33.255307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:33.256857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.257048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.257176 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c854b062-68ff-4ab5-87d6-62227b915b10, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.257288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.257390 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.257487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.257644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.257720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.257824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.257899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.258388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:33.258388 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.258388 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:33.258388 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:33.258388 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.258388 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:33.258388 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.258388 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:33.259447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.259841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:33.260036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.260036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:33.260496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:33.260847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.261204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:33.261204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.261204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:33.261672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:33.261672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:33.261804 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:33.261874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:33.262211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.262405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.262849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:33.262849 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.262849 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:33.262849 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:33.262849 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.262849 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:33.262849 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.262849 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:33.263633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:33.263926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:33.263926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:33.263926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.263926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:33.263926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:33.263926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.263926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:33.263926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.263926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.264500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:33.264771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:33.264978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.264978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:33.264978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.264978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:33.265576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:33.266003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:33.266292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.266292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:33.266292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:33.266292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.266292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:33.266292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.266292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:33.271047 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:33.271558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:33.271677 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:33.271833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:33.272217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.272436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.273006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:33.273006 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.273006 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:33.273006 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:33.273006 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.273006 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:33.273006 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.273006 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:33.273996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:33.274295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:33.274295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:33.274295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.274295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:33.274295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:33.274295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.274295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:33.274295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.274295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.274893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:33.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:33.275360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.275360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:33.275360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.275360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.275915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:33.276250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:33.276535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.276535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:33.276535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.276535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:33.280231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:33.280513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:33.280706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:33.280931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:33.281313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.281494 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:33.281765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.282436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:33.282436 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.282436 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:33.282436 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:33.282436 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.282436 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:33.282436 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.282436 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:33.283659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.283952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:33.284180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.284180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:33.284427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:33.284841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:33.285209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:33.285209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:33.285209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:37.739142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:37.740962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:37.741066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:37.741786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:37.779020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:37.779294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:37.779891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:37.779891 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:37.779891 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:37.779891 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:37.779891 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:37.779891 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:37.779891 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:37.779891 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:37.780874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:37.781192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:37.781409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:37.781409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:37.781663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:37.781996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:37.782353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:37.782353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:37.782353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:38.255706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:38.255953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:38.256093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:38.257518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.257726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.258162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.258162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.258162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.258342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.258406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.258517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-c854b062-68ff-4ab5-87d6-62227b915b10, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.258860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.259021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.259158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.259857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:38.259857 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.259857 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:38.259857 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:38.259857 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.259857 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:38.259857 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.259857 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:38.261048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.261559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:38.262111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:38.262492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.262989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:38.262989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:38.262989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.262989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:38.262989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.262989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:38.262989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:38.262989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:38.262989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:38.262989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:38.263324 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.263324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.263993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:38.263993 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.263993 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:38.263993 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:38.263993 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.263993 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:38.263993 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.263993 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:38.264644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:38.265122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:38.265122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:38.265122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.265122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:38.265122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:38.265122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.265122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:38.265122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.265122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.265444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:38.265804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:38.266272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.266272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:38.266272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.266272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:38.268018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:38.268453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:38.269021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.269021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:38.269021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:38.269021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.269021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:38.269021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.269021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:38.271498 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:38.272257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:38.272257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:38.272257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:38.272409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.272677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.273175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:38.273175 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.273175 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:38.273175 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:38.273175 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.273175 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:38.273175 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.273175 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:38.273894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:38.274305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:38.274305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:38.274305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.274305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:38.274305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:38.274305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.274305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:38.274305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.274305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.274847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:38.275450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:38.276011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:38.276235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.276536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:38.280666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:38.280838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:38.280982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:38.281055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:38.281683 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.282026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:38.282442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.283001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:38.283001 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.283001 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:38.283001 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:38.283001 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.283001 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:38.283001 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.283001 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:38.283780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.284099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:38.284515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:38.284954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:38.285541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:38.285541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:38.285541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:38.285541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:38.285541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:38.285541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:42.740540 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:42.743006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:42.743136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:42.743990 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:42.778651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:42.778966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:42.779671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:42.779671 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:42.779671 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:42.779671 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:42.779671 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:42.779671 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:42.779671 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:42.779671 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:42.780503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:42.780924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:42.781356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:42.781356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:42.781415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:42.781869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:42.782303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:42.782303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:42.782303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:43.257234 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:43.257918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:43.258254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:43.259138 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.260851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.260851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.260851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.260851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.260851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.262889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.262889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-c854b062-68ff-4ab5-87d6-62227b915b10, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.262889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.262889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.262889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-9190b208-1d8d-47af-81d6-e4b18ae14a21, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.263078 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:43.263443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.263766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:43.264058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.264058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:43.264212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:43.264597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.265055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:43.265055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.265055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:43.265055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:43.265055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:43.265055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:43.265055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:43.265643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.265701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.266130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:43.266130 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.266130 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:43.266130 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:43.266130 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.266130 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:43.266130 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.266130 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:43.266693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:43.267097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:43.267097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:43.267097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.267097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:43.267097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:43.267097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.267097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:43.267097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.267097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.267379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:43.267991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:43.268342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:43.268718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.269176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:43.271832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:43.272076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:43.272217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:43.272367 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:43.272929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.272929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.273400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:43.273400 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.273400 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:43.273400 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:43.273400 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.273400 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:43.273400 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.273400 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:43.274234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:43.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:43.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:43.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:43.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:43.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:43.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.274641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.275330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:43.275651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:43.276007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.276007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:43.276007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.276007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.276416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:43.276919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:43.277477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.277477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:43.277477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.277477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:43.280961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:43.281162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:43.281260 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:43.281453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:43.281813 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.282200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:43.282462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.282922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:43.282922 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.282922 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:43.282922 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:43.282922 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.282922 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:43.282922 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.282922 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:43.283852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.284201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:43.284516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.284516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:43.284944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:43.285603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:43.286093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:43.286093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:43.286093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:47.743864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:47.744422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:47.744422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:47.744943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:47.805820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:47.806028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:47.806662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:47.806662 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:47.806662 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:47.806662 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:47.806662 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:47.806662 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:47.806662 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:47.806662 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.062 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:47.807880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:47.808174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:47.808410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:47.808410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:47.809024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:47.809789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:47.810401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:47.810401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:47.810401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:48.258735 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:48.259041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:48.259161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:48.260342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.260487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.260642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.260775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.260866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.260966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.261075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.261225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-c854b062-68ff-4ab5-87d6-62227b915b10, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.261376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.261524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.261698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.261840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.262014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-9190b208-1d8d-47af-81d6-e4b18ae14a21, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.262174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.262332 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.262488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.263169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:48.263169 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.263169 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:48.263169 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:48.263169 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.263169 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:48.263169 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.263169 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:48.264101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.264522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:48.264876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.264876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:48.265293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:48.265642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.266051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:48.266051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.266051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:48.266051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:48.266239 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:48.266239 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:48.266386 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:48.266849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.267072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.267779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:48.267779 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.267779 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:48.267779 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:48.267779 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.267779 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:48.267779 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.267779 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:48.268638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:48.269097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:48.269097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:48.269097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.269097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:48.269097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:48.269097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.269097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:48.269097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.269097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.269537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:48.269849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:48.270087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.270087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:48.270087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.270087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:48.271274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:48.271805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:48.272103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.272103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:48.272103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:48.272103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.272103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:48.272103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.272103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:48.272548 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:48.272638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:48.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:48.272923 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:48.273836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.274161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.275472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:48.275472 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.275472 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:48.275472 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:48.275472 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.275472 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:48.275472 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.275472 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:48.276136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:48.276393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:48.276393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:48.276393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.276393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:48.276393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:48.276393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.276393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:48.276393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.276393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.276930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:48.277147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:48.277511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.277511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:48.277511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.277511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.277934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:48.278251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:48.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:48.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:48.282965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:48.282965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:48.282965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:48.282965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:48.282965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.282965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:48.283138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.283377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:48.283377 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.283377 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:48.283377 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:48.283377 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.283377 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:48.283377 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.283377 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:48.284208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.284556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:48.284948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.284948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:48.285241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:48.285531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:48.285885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:48.285885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:48.285885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:52.744284 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:52.745019 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:52.745181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:52.746014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:52.747681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=240 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:00:52.814968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:52.815254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.071 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:52.817192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:52.818011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:52.818464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:52.818902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:52.819482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:52.819482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:52.819482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:52.819482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:52.819482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:52.819482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:52.819482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:52.819482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:52.819482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:53.259160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:53.259721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:53.259976 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:53.262272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.262520 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.262702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.263288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.263474 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.263635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.263771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.263930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-c854b062-68ff-4ab5-87d6-62227b915b10, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.264071 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.264180 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.264280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.264378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.264478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-9190b208-1d8d-47af-81d6-e4b18ae14a21, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.264602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.264718 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.264883 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=15): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.264924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=16): DelStaticRoutesCommand(_result=None, lrouter=neutron-a940eef2-dfbb-4b49-8482-6a6da2849ade, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.265034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.265594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:53.265594 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.265594 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:53.265594 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:53.265594 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.265594 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:53.265594 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.265594 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:53.267020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.267280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:53.267531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.267531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:53.268275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:53.268712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.269215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:53.269215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.269215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:53.269215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:53.269215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:53.269215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:53.269215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:53.270016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.270184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.270960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:53.270960 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.270960 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:53.270960 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:53.270960 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.270960 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:53.270960 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.270960 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:53.271640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:53.272124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:53.272124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:53.272124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.272124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:53.272124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:53.272124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.272124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:53.272124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.272124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.272387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:53.272975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:53.273202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.273202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:53.273202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.273202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:53.273419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:53.273879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:53.274228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.274228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:53.274228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:53.274228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.274228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:53.274228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.274228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:53.274228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:53.274228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:53.274228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:53.274585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:53.274585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.274842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.275347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:53.275347 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.275347 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:53.275347 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:53.275347 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.275347 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:53.275347 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.275347 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:53.276093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:53.276340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:53.276340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:53.276340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.276340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:53.276340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:53.276340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.276340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:53.276340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.276340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.276951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:53.277163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:53.277418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.277418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:53.277418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.277418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.277749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:53.278091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:53.278452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.278452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:53.278452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.278452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:53.281678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:53.282041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:53.282160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:53.282334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:53.282699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.282938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:53.283298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.283868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:53.283868 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.283868 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:53.283868 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:53.283868 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.283868 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:53.283868 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.283868 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:53.284694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.284952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:53.285199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.285199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:53.285440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:53.286018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:53.286563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:53.286563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:53.286563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:57.745127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:57.745507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:57.745559 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:57.746219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:57.782291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:57.782488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:57.783001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:57.783001 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:57.783001 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:57.783001 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:57.783001 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:57.783001 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:57.783001 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:57.783001 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:57.783791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:57.784194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:57.784608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:57.784608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:57.784884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:57.785222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:57.785546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:57.785546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:57.785546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:58.259829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:58.261283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:58.261283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:58.261829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.261989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.262132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.262248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.262393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.262532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.262670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.262670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.262823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.262894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.262961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.263037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-9190b208-1d8d-47af-81d6-e4b18ae14a21, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.263134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.263265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.263362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.263459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=15): DelStaticRoutesCommand(_result=None, lrouter=neutron-a940eef2-dfbb-4b49-8482-6a6da2849ade, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.263559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.264461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:58.264461 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.264461 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:58.264461 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:58.264461 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.264461 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:58.264461 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.264461 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:58.265126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.265652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:58.266695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:58.266962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.267265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:58.267265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:58.267265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.267265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:58.267265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.267265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:58.268747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:58.269040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:58.269157 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:58.269301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:00:58.269573 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.269792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.270584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:58.270584 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.270584 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:58.270584 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:58.270584 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.270584 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:58.270584 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.270584 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:58.271034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:58.271501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:58.271501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:58.271501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.271501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:58.271501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:58.271501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.271501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:58.271501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.271501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.272038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:58.272492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:58.272768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:58.273019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.273381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:58.274814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:58.274814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:58.274814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:58.274814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:58.275117 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.275219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.276155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:58.276155 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.276155 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:58.276155 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:58.276155 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.276155 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:58.276155 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.276155 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:00:58.276857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:00:58.277241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:58.277241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:00:58.277241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.277241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:00:58.277241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:00:58.277241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.277241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:00:58.277241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.277241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.277924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:58.278447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:58.279371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:58.280043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.280689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:00:58.283945 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:00:58.284360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:00:58.284504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:00:58.284693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:00:58.284994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.285359 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:00:58.285715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.286430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:00:58.286430 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.286430 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:00:58.286430 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:00:58.286430 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.286430 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:00:58.286430 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.286430 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:00:58.286766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.287131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:00:58.287373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.287373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:00:58.287786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:00:58.288799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:00:58.289287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:00:58.289287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:00:58.289287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:02.997931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:02.997931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:02.997931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:02.997931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:02.997931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:02.997931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.070 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:03.000511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:03.001755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:03.002457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:03.003126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:03.003652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.003652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:03.003652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.003652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:03.003652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:03.003652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.003652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:03.003652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.003652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:03.262643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:03.262643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:03.262643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:03.263794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.264097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.264235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.264504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.264683 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.264862 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.264982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.265132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.265274 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.265427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.265571 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.265785 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.265877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.266018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.266167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.269344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:03.269344 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.269344 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:03.269344 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:03.269344 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.269344 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:03.269344 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.269344 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.270661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.270985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:03.272799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:03.273149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:03.273811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:03.274160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.274160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.274160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:03.274160 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.274160 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:03.274160 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:03.274160 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.274160 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:03.274160 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.274160 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:03.274841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:03.275275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.275275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:03.275275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.275275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:03.275275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:03.275275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.275275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:03.275275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.275275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:03.275275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.276472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:03.277046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:03.277360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.277675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:03.278064 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.278301 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:03.278550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:03.278794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.278794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:03.278794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.278794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:03.278794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:03.278794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.278794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:03.278794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.278794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.279490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:03.279767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:03.280074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.280074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:03.280074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.280074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.280299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:03.280753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:03.281163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.281163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:03.281163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.281163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:03.287467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:03.287976 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:03.288104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:03.288304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:03.288637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.288854 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:03.289196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.289688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:03.289688 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.289688 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:03.289688 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:03.289688 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.289688 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:03.289688 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.289688 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.290326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.290708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:03.291004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.291004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:03.291200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:03.291539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:03.291914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:03.291914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:03.291914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:07.746183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:07.746703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:07.746703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:07.747102 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:07.786703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:07.786950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:07.787447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:07.787447 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:07.787447 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:07.787447 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:07.787447 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:07.787447 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:07.787447 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:07.787447 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:07.788100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:07.788567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:07.789000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:07.789436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:07.789980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:07.789980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:07.789980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:07.789980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:07.789980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:07.789980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:08.261965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:08.262400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:08.263136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:08.264567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.264567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.264567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.264567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.264567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.264899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.264899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.264899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.264899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.265042 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.265111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.265154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.265374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.265486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.265945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:08.265945 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.265945 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:08.265945 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:08.265945 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.265945 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:08.265945 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.265945 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:08.266722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.267105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:08.267314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:08.267725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.268052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:08.268052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:08.268052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.268052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:08.268052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.268052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:08.272682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:08.273100 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:08.273196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:08.273306 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:08.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.274068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.274677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:08.274677 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.274677 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:08.274677 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:08.274677 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.274677 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:08.274677 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.274677 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:08.274918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:08.275439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:08.275439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:08.275439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.275439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:08.275439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:08.275439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.275439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:08.275439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.275439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.275597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:08.275846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:08.276072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.276072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:08.276072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.276072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:08.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:08.277336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:08.277674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.277674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:08.277674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:08.277674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.277674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:08.277674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.277674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:08.277674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:08.277674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:08.277674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:08.277876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:08.277913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.278122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.278507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:08.278507 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.278507 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:08.278507 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:08.278507 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.278507 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:08.278507 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.278507 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:08.279085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:08.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:08.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:08.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:08.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:08.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:08.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.279428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.279965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:08.280184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:08.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:08.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.281050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:08.281445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:08.281812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.281812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:08.281812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.281812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:08.288009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:08.288387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:08.288521 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:08.288716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:08.289011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.289315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:08.289614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.290081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:08.290081 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.290081 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:08.290081 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:08.290081 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.290081 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:08.290081 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.290081 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:08.290804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.291124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:08.291475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.291475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:08.291608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:08.292053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:08.292509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:08.292509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:08.292509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:12.747901 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:12.748319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:12.750043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:12.750043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:12.797804 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:12.798665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:12.799480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:12.799480 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:12.799480 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:12.799480 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:12.799480 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:12.799480 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:12.799480 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:12.799480 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:12.800380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:12.800784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:12.801311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:12.801311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:12.801460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:12.801954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:12.803809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:12.803809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:12.803809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:13.262660 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:13.263415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:13.263574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:13.267771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.267771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.267771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.267771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.267771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.267968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.267968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.267968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.267968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.267968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.268083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.268083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.268083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.268083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.268083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-39cb580d-c4cd-42c0-aaa1-f1df2139ab55, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.268083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.268187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:13.268187 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.268187 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:13.268187 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:13.268187 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.268187 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:13.268187 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.268187 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:13.268923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.269194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:13.269527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.269527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:13.271526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:13.271890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.272248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:13.272248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.272248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:13.273087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:13.273471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.276748 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:13.277278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:13.277580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:13.277580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:13.277580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.277580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:13.277580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:13.277580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.277580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:13.277580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.277580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.278237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:13.278440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:13.278697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.278697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:13.278697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.278697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:13.279887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:13.280162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:13.280424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.280424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:13.280424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:13.280424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.280424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:13.280424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.280424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:13.280703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:13.281053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:13.281207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:13.281319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:13.281853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.282177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.282814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:13.282814 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.282814 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:13.282814 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:13.282814 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.282814 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:13.282814 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.282814 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:13.283714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:13.284133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:13.284133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:13.284133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.284133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:13.284133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:13.284133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.284133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:13.284133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.284133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.284594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:13.284932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:13.285370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.285370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:13.285370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.285370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.285594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:13.286097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:13.286570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.286570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:13.286570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.286570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:13.288430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:13.288634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:13.288786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:13.288972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:13.289442 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.289665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:13.290054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.291272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:13.291272 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.291272 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:13.291272 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:13.291272 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.291272 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:13.291272 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.291272 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:13.291614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.291974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:13.292340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.292340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:13.294771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:13.295227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:13.295743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:13.295743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:13.295743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:17.748317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:17.749746 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:17.749746 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:17.750251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:17.809704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:17.810084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:17.810315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:17.810534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:17.810857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:18.263563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:18.265754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:18.265890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:18.269522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.269522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.269522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.269522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.269522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.269784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.269784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.269784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.269784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.269784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.270040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.270040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.270040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.270040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-39cb580d-c4cd-42c0-aaa1-f1df2139ab55, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.270040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.270192 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=15): DelStaticRoutesCommand(_result=None, lrouter=neutron-7b0cc6ce-2a66-437f-9ff4-2f7376455434, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.270192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:18.271513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:18.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:18.278310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:18.278877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:18.279378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.279757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.280089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:18.280515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:18.282712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:18.284036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.285123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.285540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:18.286050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:18.286050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:18.286050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:18.286050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:18.286050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.286050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.286470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:18.286470 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.286470 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:18.286470 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:18.286470 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.286470 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:18.286470 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.286470 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:18.287367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:18.287608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:18.287608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:18.287608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.287608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:18.287608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:18.287608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.287608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:18.287608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.287608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.288343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:18.288688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:18.289070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.289070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:18.289070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.289070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.289795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:18.290163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:18.290471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.290471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:18.290471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.290471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:18.290471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:18.290471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:18.290471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:18.290704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:18.291075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.291306 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:18.291856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.292322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:18.292322 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.292322 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:18.292322 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:18.292322 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.292322 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:18.292322 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.292322 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:18.293357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.293558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:18.293821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.293821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:18.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:18.294889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:18.295367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:18.295367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:18.295367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:22.748885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:22.749345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:22.749430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:22.750101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:22.789545 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:22.789874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:22.790482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:22.790482 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:22.790482 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:22.790482 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:22.790482 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:22.790482 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:22.790482 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:22.790482 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:22.791229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:22.791704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:22.792090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:22.792537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:22.793098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:22.793098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:22.793098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:22.793098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:22.793098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:22.793098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:23.265875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:23.266125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:23.266231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:23.267454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.267712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.267896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.268063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.268217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.268364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.268475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.268584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.268765 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.268925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.269089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.269239 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.269401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.269558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-39cb580d-c4cd-42c0-aaa1-f1df2139ab55, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.269748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.269909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=15): DelStaticRoutesCommand(_result=None, lrouter=neutron-7b0cc6ce-2a66-437f-9ff4-2f7376455434, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.270050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.270505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:23.270505 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.270505 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:23.270505 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:23.270505 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.270505 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:23.270505 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.270505 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:23.271170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.271425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:23.271708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.271708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:23.271904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:23.272230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:23.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.272538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:23.273986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:23.274218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:23.274311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:23.274407 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:23.274772 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.274985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.275475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:23.275475 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.275475 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:23.275475 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:23.275475 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.275475 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:23.275475 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.275475 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:23.276000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:23.276297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:23.276297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:23.276297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.276297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:23.276297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:23.276297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.276297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:23.276297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.276297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.276559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:23.276837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:23.277063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.277063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:23.277063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.277063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:23.277310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:23.277741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:23.278081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.278081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:23.278081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:23.278081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.278081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:23.278081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.278081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:23.282823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:23.283075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:23.283164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:23.283278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:23.283648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.283918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.284497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:23.284497 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.284497 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:23.284497 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:23.284497 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.284497 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:23.284497 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.284497 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:23.285046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:23.285363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:23.285363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:23.285363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.285363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:23.285363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:23.285363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.285363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:23.285363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.285363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.285660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:23.285876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:23.286162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.286162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:23.286162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.286162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.286343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:23.286654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:23.286956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.286956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:23.286956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.286956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:23.290591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:23.290854 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:23.290940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:23.291068 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:23.291391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.291689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:23.292177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.292701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:23.292701 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.292701 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:23.292701 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:23.292701 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.292701 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:23.292701 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.292701 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:23.293302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.293540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:23.293798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.293798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:23.293964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:23.294279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:23.294596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:23.294596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:23.294596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:27.749368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:27.751799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:27.751799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:27.751799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:27.824132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:27.824295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:27.825007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:27.825007 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:27.825007 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:27.825007 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:27.825007 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:27.825007 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:27.825007 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:27.825007 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.075 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:27.826901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:27.827170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:27.827396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:27.827396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:27.827958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:27.828264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:27.828585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:27.828585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:27.828585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:28.266487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:28.267185 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:28.267185 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:28.269321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.270528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.270777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.270934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.271808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.272016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.272168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.272277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.272419 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.273924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.274076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.274514 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.274584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:28.274922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.275129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-39cb580d-c4cd-42c0-aaa1-f1df2139ab55, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.275263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.275409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=15): DelStaticRoutesCommand(_result=None, lrouter=neutron-7b0cc6ce-2a66-437f-9ff4-2f7376455434, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.275543 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=16): DelStaticRoutesCommand(_result=None, lrouter=neutron-8359da02-7262-4819-920f-5f7ff5bee651, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.277093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.277721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:28.277721 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.277721 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:28.277721 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:28.277721 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.277721 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:28.277721 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.277721 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:28.278892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.279145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:28.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:28.280276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:28.280547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.280845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:28.280845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.280845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:28.280845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:28.280845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:28.280845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:28.281679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.281679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.282483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:28.282483 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.282483 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:28.282483 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:28.282483 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.282483 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:28.282483 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.282483 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:28.283028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:28.283326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:28.283326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:28.283326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.283326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:28.283326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:28.283326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.283326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:28.283326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.283326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.284202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:28.284457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:28.284705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.284705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:28.284705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.284705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:28.284705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:28.285743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:28.287799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:28.288564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.288564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:28.288564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:28.288564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.288564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:28.288564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.288564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:28.288564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:28.288564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:28.288564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:28.288763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.288973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:28.289283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.289523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:28.289781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:28.289781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.289781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:28.289781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.289781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.292945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:28.293207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:28.293526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.293526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:28.293526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.293526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:28.293526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:28.293526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:28.293526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:28.293526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.293697 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:28.294122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.294383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:28.294689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.294689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:28.295173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:28.295510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:28.295865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:28.295865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:28.295865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:32.749888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:32.750405 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:32.750405 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:32.751038 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:32.815182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:32.815182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:32.815182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:32.815182 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:32.815182 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:32.815182 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:32.815182 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:32.815182 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:32.815182 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:32.815182 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.065 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:32.816551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:32.816904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:32.817283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:32.817283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:32.817347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:32.818806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:32.819347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:32.819347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:32.819347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:33.267888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:33.268464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:33.269138 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:33.273630 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.273838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-39cb580d-c4cd-42c0-aaa1-f1df2139ab55, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-7b0cc6ce-2a66-437f-9ff4-2f7376455434, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-8359da02-7262-4819-920f-5f7ff5bee651, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=15): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=16): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.275787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.277129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:33.277129 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.277129 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:33.277129 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:33.277129 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.277129 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:33.277129 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.277129 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:33.280653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.281174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:33.281479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.281479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:33.282255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:33.282726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.283225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:33.283225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.283225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:33.283508 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:33.284982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:33.285103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:33.285571 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:33.286003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.287004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.289349 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:33.289666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:33.289666 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.289666 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:33.289666 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:33.289666 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.289666 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:33.289666 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.289666 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:33.291025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:33.291511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:33.291511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:33.291511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.291511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:33.291511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:33.291511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.291511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:33.291511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.291511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.293786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:33.294233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:33.294655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.294655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:33.294655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.294655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:33.294655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:33.299439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:33.300484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:33.301208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.301208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:33.301208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:33.301208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.301208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:33.301208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.301208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:33.301208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:33.301208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:33.301208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:33.301501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.301501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.303215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:33.303215 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.303215 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:33.303215 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:33.303215 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.303215 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:33.303215 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.303215 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:33.303789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:33.304165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:33.304165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:33.304165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.304165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:33.304165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:33.304165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.304165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:33.304165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.304165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.306676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:33.307152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:33.307499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.307499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:33.307499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.307499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.308736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:33.309572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:33.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:33.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:33.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:33.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:33.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:33.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.310327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:33.310327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.310880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:33.310880 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.310880 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:33.310880 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:33.310880 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.310880 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:33.310880 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.310880 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:33.312714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.313093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:33.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:33.314527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:33.315457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:33.316376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:33.316376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:33.316376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:37.750684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:37.751183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:37.751183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:37.751954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:37.814396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:37.814753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.064 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:37.816905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:37.817187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:37.817427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:37.817656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:37.817961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:37.817961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:37.817961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:37.817961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:37.817961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:37.817961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:37.817961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:37.817961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:37.817961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:38.268919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:38.270205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:38.270205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:38.274376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.275088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.275088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.275088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.276717 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.276717 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.276717 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.276717 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.276717 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-39cb580d-c4cd-42c0-aaa1-f1df2139ab55, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.276986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.280680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-7b0cc6ce-2a66-437f-9ff4-2f7376455434, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.280680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-8359da02-7262-4819-920f-5f7ff5bee651, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.280680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.280680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.280680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.280904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=15): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.281711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=16): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.281711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.282135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:38.282135 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.282135 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:38.282135 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:38.282135 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.282135 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:38.282135 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.282135 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:38.283263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.283584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:38.284057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.284057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:38.285222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:38.285693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.287226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:38.287226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.287226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:38.287226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:38.287226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:38.287226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:38.287226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:38.287226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.287461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.287461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:38.287461 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.287461 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:38.287461 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:38.287461 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.287461 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:38.287461 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.287461 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:38.288272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:38.288730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:38.288730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:38.288730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.288730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:38.288730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:38.288730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.288730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:38.288730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.288730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.289093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:38.289477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:38.289910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.289910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:38.289910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.289910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:38.292381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:38.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:38.293825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.293825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:38.293825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:38.293825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.293825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:38.293825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.293825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:38.300651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:38.300651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:38.300651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:38.300651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:38.302642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.304934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.307644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:38.308177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:38.308569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.308919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:38.309255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.309255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.310753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:38.311115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:38.311401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.311401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:38.311401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.311401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:38.311401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:38.311401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:38.311510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:38.311911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:38.313867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.314406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:38.315709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:38.319666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:38.320207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.320422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:38.320422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:38.320422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.320422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:38.320422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.320422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:38.322925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:38.323417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:38.323866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:38.323866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:38.323866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:42.751423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:42.751423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:42.751820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:42.752297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:42.819212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:42.819393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:42.819991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:42.819991 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:42.819991 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:42.819991 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:42.819991 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:42.819991 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:42.819991 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:42.819991 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.068 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:42.821238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:42.821639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:42.822135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:42.822135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:42.822452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:42.823005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:42.823523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:42.823523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:42.823523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:43.270835 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:43.270835 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:43.270835 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:43.273423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.273423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.273589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.274046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.274046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.274122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.274237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.274373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.274513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-39cb580d-c4cd-42c0-aaa1-f1df2139ab55, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.274673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.274915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-7b0cc6ce-2a66-437f-9ff4-2f7376455434, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.276617 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-8359da02-7262-4819-920f-5f7ff5bee651, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.276617 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.276617 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.276617 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.277447 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=15): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.277447 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=16): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.277447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.278406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:43.278406 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.278406 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:43.278406 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:43.278406 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.278406 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:43.278406 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.278406 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:43.279305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.279691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:43.280012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.280012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:43.280324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:43.280756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.281186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:43.281186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.281186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:43.286859 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:43.286859 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:43.286859 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:43.286859 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:43.286859 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.287134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.287806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:43.287806 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.287806 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:43.287806 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:43.287806 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.287806 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:43.287806 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.287806 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:43.288686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:43.289245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:43.289245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:43.289245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.289245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:43.289245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:43.289245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.289245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:43.289245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.289245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.289644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:43.289996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:43.290309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.290309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:43.290309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.290309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:43.290862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:43.291417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:43.292103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.292103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:43.292103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:43.292103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.292103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:43.292103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.292103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:43.298770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:43.299220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:43.299333 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:43.299490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:43.299926 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.300171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.301601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:43.301601 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.301601 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:43.301601 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:43.301601 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.301601 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:43.301601 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.301601 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:43.302422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:43.302837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:43.302837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:43.302837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.302837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:43.302837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:43.302837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.302837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:43.302837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.302837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.303818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:43.304180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:43.304570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:43.304982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.305467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:43.311560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:43.312004 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:43.312128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:43.312300 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:43.312967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.313580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:43.314595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:43.316985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:43.317540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:43.317889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:43.318195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:43.318715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:43.318715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:43.318715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:43.318715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:43.318715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:43.318715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:43.318715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:43.318715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:43.318715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:43.318715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:47.751512 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:47.752062 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:47.752783 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:47.754599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:47.807319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:47.807517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:47.808166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:47.808166 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:47.808166 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:47.808166 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:47.808166 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:47.808166 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:47.808166 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:47.808166 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.056 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:47.810788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:47.811214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:47.811495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:47.812720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:47.813662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:47.813662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:47.813662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:47.813662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:47.813662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:47.813662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:48.270577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:48.271166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:48.271166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:48.272579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.273038 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.273418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.273719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.273936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.274160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.274403 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.274866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.275106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-39cb580d-c4cd-42c0-aaa1-f1df2139ab55, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.275675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.275675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-7b0cc6ce-2a66-437f-9ff4-2f7376455434, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.275900 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-8359da02-7262-4819-920f-5f7ff5bee651, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.276178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.276374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.276540 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.276726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=15): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.276880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=16): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.277024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.278168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:48.278168 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.278168 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:48.278168 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:48.278168 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.278168 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:48.278168 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.278168 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:48.279241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.279706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:48.280076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.280076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:48.280160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:48.280610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.281137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:48.281137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.281137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:48.286117 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:48.286431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:48.286529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:48.286701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:48.287084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.287421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.288127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:48.288127 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.288127 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:48.288127 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:48.288127 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.288127 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:48.288127 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.288127 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:48.288746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:48.289164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:48.289164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:48.289164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.289164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:48.289164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:48.289164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.289164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:48.289164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.289164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.289456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:48.289862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:48.290262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:48.290785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.291190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:48.299274 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:48.299772 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:48.299924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:48.300081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:48.300743 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.301023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.302051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:48.302051 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.302051 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:48.302051 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:48.302051 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.302051 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:48.302051 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.302051 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:48.303108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:48.303384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:48.303384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:48.303384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.303384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:48.303384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:48.303384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.303384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:48.303384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.303384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.304414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:48.304747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:48.305136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.305136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:48.305136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.305136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.305435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:48.305901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:48.306411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.306411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:48.306411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.306411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:48.312052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:48.312337 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:48.312453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:48.312682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:48.313278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.313576 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:48.314139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.314895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:48.314895 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.314895 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:48.314895 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:48.314895 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.314895 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:48.314895 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.314895 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:48.315948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:48.316685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.316685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:48.317091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:48.317515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:48.317986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:48.317986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:48.317986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:52.752067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:52.752560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:52.752777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:52.753920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:52.762771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=241 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:01:52.811230 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.059 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:52.812979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:52.813417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:52.813737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:52.814052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:52.814515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:53.271038 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:53.272421 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:53.272421 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:53.273216 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.273468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.273612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.273787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.273929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.274138 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.274317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.274434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.274533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-39cb580d-c4cd-42c0-aaa1-f1df2139ab55, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.274646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.274768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-7b0cc6ce-2a66-437f-9ff4-2f7376455434, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.274977 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-8359da02-7262-4819-920f-5f7ff5bee651, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.275083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.275183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.275278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.275386 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=15): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.275603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=16): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.275901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.276313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:53.276313 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.276313 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:53.276313 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:53.276313 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.276313 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:53.276313 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.276313 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:53.277052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.277378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:53.277651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.277651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:53.277762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:53.278179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.278492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:53.278492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.278492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:53.286882 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:53.287262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:53.287369 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:53.287466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:53.287957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.288216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.288833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:53.288833 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.288833 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:53.288833 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:53.288833 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.288833 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:53.288833 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.288833 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:53.289459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:53.289766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:53.289766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:53.289766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.289766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:53.289766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:53.289766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.289766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:53.289766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.289766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.290124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:53.290794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:53.291299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:53.291545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.291828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:53.299732 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:53.300044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:53.300097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:53.300204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:53.300777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.301067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.301691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:53.301691 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.301691 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:53.301691 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:53.301691 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.301691 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:53.301691 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.301691 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:53.302531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:53.302925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:53.302925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:53.302925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.302925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:53.302925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:53.302925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.302925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:53.302925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.302925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.303261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:53.303572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:53.303934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.303934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:53.303934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.303934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.304605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:53.305678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:53.306721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.306721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:53.306721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.306721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:53.312534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:53.312914 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:53.312998 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:53.313170 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:53.313598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.313889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:53.314338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.314930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:53.314930 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.314930 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:53.314930 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:53.314930 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.314930 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:53.314930 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.314930 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:53.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.316215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:53.316566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.316566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:53.316982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:53.317386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:53.317871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:53.317871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:53.317871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:57.752575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:57.753181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:57.753234 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:57.754039 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:57.822000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:57.822260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:57.822921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:57.822921 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:57.822921 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:57.822921 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:57.822921 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:57.822921 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:57.822921 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:57.822921 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.070 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:57.823611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:57.823958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:57.824390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:57.824390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:57.824468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:57.824916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:57.825469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:57.825469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:57.825469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:58.271563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:58.272036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:58.272180 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:58.273882 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.274150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.274312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.274470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.274635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.274794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.274944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.275090 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.275252 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.275387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.275526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.277380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.277696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.277865 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.277983 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-8359da02-7262-4819-920f-5f7ff5bee651, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.278102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.278809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:58.278809 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.278809 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:58.278809 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:58.278809 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.278809 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:58.278809 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.278809 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:58.280220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.280671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:58.281127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.281127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:58.281309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:58.281779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.282353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:58.282353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.282353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:58.287443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:58.287936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:58.288057 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:58.288195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:01:58.288682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.288900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.289460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:58.289460 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.289460 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:58.289460 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:58.289460 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.289460 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:58.289460 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.289460 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:58.290268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:58.290710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:58.290710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:58.290710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.290710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:58.290710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:58.290710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.290710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:58.290710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.290710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.291106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:58.291408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:58.291765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.291765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:58.291765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.291765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:58.292146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:58.292561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:58.293060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.293060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:58.293060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:58.293060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.293060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:58.293060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.293060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:58.300279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:58.300833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:58.300956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:58.301204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:58.302298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.302465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.303894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:58.303894 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.303894 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:58.303894 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:58.303894 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.303894 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:58.303894 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.303894 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:01:58.305273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:01:58.305575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:58.305575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:01:58.305575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.305575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:01:58.305575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:01:58.305575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.305575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:01:58.305575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.305575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.305963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:58.306282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:58.306510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.306510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:58.306510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.306510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.307888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:58.308322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:58.308644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.308644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:58.308644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.308644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:01:58.313144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:01:58.313458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:01:58.313563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:01:58.313747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:01:58.314144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.314399 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:01:58.314793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.315297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:01:58.315297 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.315297 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:01:58.315297 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:01:58.315297 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.315297 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:01:58.315297 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.315297 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:01:58.316348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.316671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:01:58.316925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.316925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:01:58.317433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:01:58.317765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:01:58.318184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:01:58.318184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:01:58.318184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:02.753121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:02.753745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:02.753745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:02.754344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:02.816254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:02.816683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:02.817312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:02.817312 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:02.817312 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:02.817312 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:02.817312 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:02.817312 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:02.817312 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:02.817312 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.064 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:02.818423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:02.818870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:02.819557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:02.820103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:02.820671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:02.820671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:02.820671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:02.820671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:02.820671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:02.820671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:03.273217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:03.273217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:03.273217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:03.274138 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.274417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.274637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.274858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.275057 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.275236 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.275400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.275751 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.276009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.276876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.276876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.276876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.276876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.277111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.277238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.278209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:03.278209 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.278209 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:03.278209 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:03.278209 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.278209 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:03.278209 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.278209 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:03.278863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.280474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:03.280708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:03.280955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.281242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:03.281242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:03.281242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.281242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:03.281242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.281242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:03.287935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:03.288390 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:03.288524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:03.289649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:03.289649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.289649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.289954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:03.289954 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.289954 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:03.289954 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:03.289954 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.289954 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:03.289954 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.289954 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:03.290808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:03.291108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:03.291108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:03.291108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.291108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:03.291108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:03.291108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.291108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:03.291108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.291108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.291710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:03.292914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:03.293271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:03.295722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.296258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:03.300810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:03.301140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:03.301382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:03.301382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:03.301795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.302031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.302600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:03.302600 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.302600 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:03.302600 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:03.302600 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.302600 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:03.302600 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.302600 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:03.303657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:03.304257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:03.304679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:03.305141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:03.305469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:03.315145 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:03.315606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:03.315738 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:03.315994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:03.316958 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.316958 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:03.317604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.318315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:03.318315 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.318315 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:03.318315 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:03.318315 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.318315 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:03.318315 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.318315 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:03.319759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.320179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:03.320485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.320485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:03.321415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:03.321992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:03.322299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:03.322299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:03.322299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:07.753589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:07.754075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:07.754120 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:07.754953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:07.821853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:07.822159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:07.822669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:07.822669 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:07.822669 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:07.822669 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:07.822669 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:07.822669 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:07.822669 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:07.822669 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.069 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:07.823493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:07.823902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:07.824209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:07.824209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:07.824538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:07.824988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:07.825801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:07.825801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:07.825801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:08.272659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:08.273129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:08.273269 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:08.274797 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.275103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f47b502-cf94-4d11-a8ae-cb76bd1cd980, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.275310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.275515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.275669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.275880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.275936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.276084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.276220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.276406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.276501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.276659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.276803 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.276938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.277072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-079b90c5-3c89-42c4-938b-d25f8ad92d80, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.277214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.279883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:08.279883 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.279883 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:08.279883 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:08.279883 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.279883 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:08.279883 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.279883 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:08.280804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.281232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:08.281651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.281651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:08.281938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:08.282385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.282846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:08.282846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.282846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:08.288463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:08.288897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:08.289013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:08.289143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:08.289967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.290138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.290650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:08.290650 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.290650 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:08.290650 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:08.290650 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.290650 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:08.290650 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.290650 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:08.291398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:08.291827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:08.291827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:08.291827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.291827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:08.291827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:08.291827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.291827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:08.291827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.291827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.292093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:08.292405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:08.292826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.292826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:08.292826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.292826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:08.292969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:08.293457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:08.294030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.294030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:08.294030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:08.294030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.294030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:08.294030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.294030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:08.301919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:08.306821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:08.306972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:08.307066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:08.307557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.307810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.308278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:08.308278 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.308278 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:08.308278 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:08.308278 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.308278 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:08.308278 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.308278 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:08.309029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:08.309705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:08.309705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:08.309705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.309705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:08.309705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:08.309705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.309705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:08.309705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.309705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.309978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:08.310289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:08.310720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:08.311122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.311691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:08.315563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:08.316940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:08.317048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:08.317213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:08.318731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.318816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:08.319329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.323598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:08.323598 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.323598 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:08.323598 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:08.323598 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.323598 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:08.323598 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.323598 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:08.324447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.324951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:08.325298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:08.325753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:08.326195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:08.326195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:08.326195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:08.326195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:08.326195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:08.326195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:12.754160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:12.754890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:12.754890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:12.755513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:12.822434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:12.822715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:12.823316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:12.823316 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:12.823316 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:12.823316 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:12.823316 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:12.823316 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:12.823316 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:12.823316 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.069 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:12.823974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:12.824271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:12.824499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:12.824499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:12.824792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:12.825060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:12.825341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:12.825341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:12.825341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:13.273172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:13.273502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:13.273636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:13.275158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.275433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.275635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.275816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.275966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.276108 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.276243 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.276373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.276567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.276751 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.276881 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.277006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.277130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.277256 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-079b90c5-3c89-42c4-938b-d25f8ad92d80, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.277380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.278031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:13.278031 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.278031 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:13.278031 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:13.278031 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.278031 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:13.278031 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.278031 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:13.279028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.279331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:13.279592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.279592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:13.280055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:13.280382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.280770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:13.280770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.280770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:13.289023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:13.289412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:13.289535 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:13.289664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:13.290130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.290384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.291016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:13.291016 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.291016 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:13.291016 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:13.291016 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.291016 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:13.291016 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.291016 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:13.291891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:13.292217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:13.292217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:13.292217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.292217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:13.292217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:13.292217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.292217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:13.292217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.292217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.292857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:13.293130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:13.293400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.293400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:13.293400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.293400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:13.293897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:13.294234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:13.294607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.294607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:13.294607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:13.294607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.294607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:13.294607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.294607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:13.302860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:13.303277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:13.303384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:13.303526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:13.304123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.304459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.305140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:13.305140 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.305140 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:13.305140 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:13.305140 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.305140 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:13.305140 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.305140 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:13.305990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:13.306283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:13.306283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:13.306283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.306283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:13.306283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:13.306283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.306283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:13.306283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.306283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.306889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:13.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:13.307398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.307398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:13.307398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.307398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.310076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:13.310540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:13.310903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.310903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:13.310903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.310903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:13.317064 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:13.317337 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:13.317468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:13.317642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:13.318149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.318411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:13.318823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.319356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:13.319356 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.319356 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:13.319356 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:13.319356 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.319356 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:13.319356 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.319356 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:13.320296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.320690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:13.321019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.321019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:13.321255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:13.321680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:13.322116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:13.322116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:13.322116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:17.754870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:17.757957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:17.758081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:17.758841 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:17.835096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:17.835437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:17.836269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:17.836269 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:17.836269 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:17.836269 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:17.836269 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:17.836269 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:17.836269 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:17.836269 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.078 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:17.837262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:17.837613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:17.837899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:17.837899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:17.838418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:17.838740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:17.839176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:17.839176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:17.839176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:18.276939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:18.277341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:18.277485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:18.279384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.280816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.281857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.282108 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.282305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.282476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.282835 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.283163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.283348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.283507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.283677 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.283833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.283979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.284251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-079b90c5-3c89-42c4-938b-d25f8ad92d80, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.284350 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.284489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.285932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:18.285932 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.285932 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:18.285932 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:18.285932 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.285932 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:18.285932 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.285932 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:18.286953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.287364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:18.287758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.287758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:18.288102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:18.288580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.289128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:18.289128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.289128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:18.289990 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:18.291890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:18.292086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:18.292351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:18.292907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.293174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.295946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:18.295946 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.295946 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:18.295946 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:18.295946 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.295946 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:18.295946 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.295946 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:18.296883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:18.297402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:18.297402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:18.297402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.297402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:18.297402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:18.297402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.297402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:18.297402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.297402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.297986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:18.298400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:18.298777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.298777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:18.298777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.298777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:18.302591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:18.303156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:18.303584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.303584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:18.303584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:18.303584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.303584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:18.303584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.303584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:18.303584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:18.303844 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:18.304024 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:18.304024 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:18.305123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.305280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.305964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:18.305964 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.305964 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:18.305964 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:18.305964 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.305964 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:18.305964 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.305964 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:18.306736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:18.307168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:18.307168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:18.307168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.307168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:18.307168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:18.307168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.307168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:18.307168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.307168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.308138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:18.308930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:18.309419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.309419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:18.309419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.309419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.311878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:18.312735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:18.313083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.313083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:18.313083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.313083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:18.317712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:18.318181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:18.318338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:18.318527 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:18.320705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.321390 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:18.323335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.325079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:18.325079 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.325079 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:18.325079 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:18.325079 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.325079 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:18.325079 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.325079 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:18.325749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.326256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:18.326783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:18.328556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:18.329510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:18.329510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:18.329510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:18.329510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:18.329510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:18.329510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:22.758510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:22.758510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:22.758510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:22.759246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:22.805704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:22.806264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:22.806522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:22.806749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:22.807085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:23.277594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:23.277723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:23.277832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:23.279065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.279065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.279271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.279271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.279366 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.279594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.279594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.279683 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.279787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.279892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.279976 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.280093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.280197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.280322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-079b90c5-3c89-42c4-938b-d25f8ad92d80, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.280408 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.280458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.280896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:23.280896 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.280896 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:23.280896 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:23.280896 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.280896 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:23.280896 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.280896 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:23.281483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.281887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:23.282287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:23.282722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.283306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:23.283306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:23.283306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.283306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:23.283306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.283306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:23.292072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:23.292771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:23.292771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:23.292771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:23.293962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.294162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.295190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:23.295190 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.295190 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:23.295190 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:23.295190 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.295190 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:23.295190 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.295190 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:23.296250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:23.296492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:23.296492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:23.296492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.296492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:23.296492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:23.296492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.296492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:23.296492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.296492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.297768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:23.297983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:23.298268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.298268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:23.298268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.298268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:23.300960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:23.301238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:23.301513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.301513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:23.301513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:23.301513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.301513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:23.301513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.301513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:23.303799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:23.304577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:23.304577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:23.304577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:23.304751 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.304942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.308723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:23.309224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:23.309646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.309975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:23.310503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:23.310883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:23.311291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:23.311291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.311291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:23.311291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:23.311291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.311291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:23.311291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.311291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:23.318303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:23.318548 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:23.318662 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:23.318816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:23.319245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.319521 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:23.320484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.321167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:23.321167 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.321167 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:23.321167 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:23.321167 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.321167 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:23.321167 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.321167 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:23.322662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.322928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:23.323138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:23.323383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:23.323689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:23.323689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:23.323689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:23.323689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:23.323689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:23.323689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:27.758568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:27.759486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:27.759548 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:27.760174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:27.832129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:27.832901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:27.833381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:27.833381 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:27.833381 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:27.833381 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:27.833381 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:27.833381 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:27.833381 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:27.833381 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.074 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:27.834680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:27.835186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:27.836307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:27.836916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:27.837368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:27.837368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:27.837368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:27.837368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:27.837368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:27.837368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:28.277864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:28.278184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:28.278275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:28.280551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-079b90c5-3c89-42c4-938b-d25f8ad92d80, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.280988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.281031 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.281182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.281644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:28.281644 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.281644 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:28.281644 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:28.281644 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.281644 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:28.281644 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.281644 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:28.282479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.282727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:28.282959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.282959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:28.283245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:28.283505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.283827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:28.283827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.283827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:28.292698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:28.292971 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:28.293169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:28.293169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:28.293580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.293826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:28.294386 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.294386 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:28.294386 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:28.294386 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.294386 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:28.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.294386 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:28.295064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:28.295355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:28.295355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:28.295355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.295355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:28.295355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:28.295355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.295355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:28.295355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.295355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.295679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:28.295955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:28.296221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.296221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:28.296221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.296221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:28.296747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:28.297180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:28.297674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.297674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:28.297674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:28.297674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.297674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:28.297674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.297674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:28.304281 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:28.304674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:28.304776 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:28.304891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:28.305255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.305480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.305979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:28.305979 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.305979 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:28.305979 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:28.305979 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.305979 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:28.305979 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.305979 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:28.306564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:28.306903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:28.306903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:28.306903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.306903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:28.306903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:28.306903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.306903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:28.306903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.306903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.307170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:28.307355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:28.307575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.307575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:28.307575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.307575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.307906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:28.308221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:28.308508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.308508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:28.308508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.308508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:28.318682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:28.320290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:28.320290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:28.320290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:28.320290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.320290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:28.320290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.320580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:28.320580 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.320580 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:28.320580 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:28.320580 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.320580 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:28.320580 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.320580 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:28.321199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.321417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:28.321659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.321659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:28.321884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:28.322270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:28.322851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:28.322851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:28.322851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:32.759663 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:32.761147 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:32.761305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:32.762220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:32.817641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:32.817928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:32.818534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:32.818534 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:32.818534 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:32.818534 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:32.818534 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:32.818534 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:32.818534 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:32.818534 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:32.819926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:32.820284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:32.821791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:32.822184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:32.822732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:32.822732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:32.822732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:32.822732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:32.822732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:32.822732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:33.281511 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:33.281910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:33.283046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:33.283763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.284035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.284220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.284416 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.285705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.286412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.286527 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.286719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.286872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.287218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.287218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.287308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-079b90c5-3c89-42c4-938b-d25f8ad92d80, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.287487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.287708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-c989e10a-10db-4b54-8ad8-b9bb5127ea20, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.289930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=14): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.290160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:33.292691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:33.293184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:33.293397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:33.293730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:33.294070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:33.295002 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:33.295002 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:33.295002 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:33.295286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.295498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.301263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:33.301770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:33.302006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.302212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:33.302430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:33.302695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:33.302981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:33.302981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:33.302981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.302981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:33.302981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:33.302981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.302981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:33.302981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.302981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:33.305781 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:33.306214 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:33.306509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:33.306957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:33.307513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.307852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.311463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:33.312119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:33.312508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.312863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:33.313204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:33.313589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:33.314059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:33.314059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.314059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:33.314059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:33.314059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.314059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:33.314059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.314059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:33.319062 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:33.319462 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:33.319729 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:33.319950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:33.320586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.320949 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:33.321464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:33.324230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:33.324872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:33.325688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:33.326075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:33.326634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:33.326634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:33.326634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:33.326634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:33.326634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:33.326634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:33.326634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:33.326634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:33.326634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:33.326634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:37.763360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:37.763360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:37.763360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:37.763360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:37.849664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:37.850399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:37.851105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:37.851105 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:37.851105 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:37.851105 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:37.851105 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:37.851105 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:37.851105 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:37.851105 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.089 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:37.852414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:37.852785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:37.853163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:37.853163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:37.854731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:37.855412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:37.855967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:37.855967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:37.855967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:38.282095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:38.282292 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:38.282434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:38.288008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.288239 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.288385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-02bbd92d-1a8c-4ab3-a0b0-93e7da19ed4b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.288533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.288696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-175df68a-c70e-47db-805c-c7a3da798e3f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.288807 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.288930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.289032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.289144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.289238 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.289345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-079b90c5-3c89-42c4-938b-d25f8ad92d80, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.289444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.289857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-234b59f2-6425-4af1-b231-1a0af59aa962, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.289857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.289857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.294979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:38.297640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:38.297640 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.297640 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:38.297640 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:38.297640 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.297640 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:38.297640 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.297640 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.015 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:38.300188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.300502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:38.301960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.301960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:38.303227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:38.303725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.304173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:38.304173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.304173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:38.304173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:38.304173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:38.304173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:38.305227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.305393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.306280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:38.306280 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.306280 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:38.306280 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:38.306280 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.306280 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:38.306280 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.306280 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:38.307749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:38.308124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:38.308124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:38.308124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.308124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:38.308124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:38.308124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.308124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:38.308124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.308124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.310123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:38.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:38.310985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:38.311254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:38.311658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.311892 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:38.312078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:38.312347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:38.312347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:38.312347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.312347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:38.312347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:38.312347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.312347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:38.312347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.312347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.312505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:38.312762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:38.312977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.312977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:38.312977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.312977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.313363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:38.313817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:38.314207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.314207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:38.314207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.314207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:38.319609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:38.320717 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:38.320967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:38.321081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:38.321429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.321677 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:38.322225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.322825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:38.322825 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.322825 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:38.322825 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:38.322825 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.322825 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:38.322825 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.322825 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:38.323903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.324272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:38.324635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.324635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:38.324987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:38.325496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:38.326091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:38.326091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:38.326091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:42.764561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:42.764561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:42.764561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:42.764561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:42.829679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:42.829926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.068 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:42.833184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:42.833907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:42.834398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:42.834838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:42.835429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:42.835429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:42.835429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:42.835429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:42.835429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:42.835429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:42.835429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:42.835429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:42.835429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:43.282547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:43.282991 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:43.283153 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:43.284832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.285031 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46fa58c2-b943-481f-9eda-37519cc587bc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.285175 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.285316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.285446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.285587 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.285715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.285831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-079b90c5-3c89-42c4-938b-d25f8ad92d80, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.285949 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.286066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.286204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-20626950-c5fb-4c00-944e-142781f0d0e0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.286357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.287058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:43.287058 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.287058 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:43.287058 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:43.287058 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.287058 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:43.287058 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.287058 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:43.287971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.288281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:43.288545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.288545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:43.288949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:43.289309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.289685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:43.289685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.289685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:43.303788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:43.304170 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:43.304284 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:43.304406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:43.305065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.305392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.307600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:43.307600 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.307600 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:43.307600 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:43.307600 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.307600 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:43.307600 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.307600 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:43.308466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:43.309025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:43.309025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:43.309025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.309025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:43.309025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:43.309025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.309025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:43.309025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.309025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.309357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:43.309910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:43.310353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:43.310796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:43.311325 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:43.311542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:43.312411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:43.314460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:43.314806 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.315001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.315430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:43.315430 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.315430 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:43.315430 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:43.315430 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.315430 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:43.315430 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.315430 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:43.316205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:43.316641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:43.316641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:43.316641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.316641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:43.316641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:43.316641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.316641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:43.316641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.316641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.317126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:43.317370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:43.317723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.317723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:43.317723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.317723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.318160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:43.318450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:43.318826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.318826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:43.318826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.318826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:43.320638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:43.326012 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:43.326148 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:43.326324 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:43.326667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.326875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:43.327191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.327785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:43.327785 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.327785 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:43.327785 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:43.327785 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.327785 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:43.327785 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.327785 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:43.328667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.329253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:43.329843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:43.330340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:43.331197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:43.331197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:43.331197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:43.331197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:43.331197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:43.331197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:47.762978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:47.763659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:47.763884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:47.764900 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:47.846050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:47.846050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:47.846050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:47.846050 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:47.846050 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:47.846050 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:47.846050 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:47.846050 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:47.846050 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:47.846050 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.082 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:47.846941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:47.847315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:47.847548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:47.847548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:47.848097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:47.848401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:47.852057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:47.852057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:47.852057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:48.283054 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:48.283823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:48.284081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:48.285495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.285770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.285985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-2831554c-b033-4517-bc26-6a0505203b4e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.286155 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.286308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.286418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.286538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-079b90c5-3c89-42c4-938b-d25f8ad92d80, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.286691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.286891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.287003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.287136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.287264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.288060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:48.288060 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.288060 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:48.288060 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:48.288060 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.288060 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:48.288060 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.288060 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:48.289073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.289356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:48.289707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.289707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:48.290129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:48.290532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.291050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:48.291050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.291050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:48.304987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:48.305426 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:48.305684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:48.305975 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:48.307526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.307742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.308311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:48.308311 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.308311 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:48.308311 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:48.308311 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.308311 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:48.308311 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.308311 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:48.309069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:48.309441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:48.309441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:48.309441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.309441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:48.309441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:48.309441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.309441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:48.309441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.309441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.309723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:48.310037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:48.310270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.310270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:48.310270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.310270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:48.310414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:48.310684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:48.313246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.313246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:48.313246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:48.313246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.313246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:48.313246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.313246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:48.313246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:48.313246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:48.313246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:48.313717 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:48.314860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.315091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.315908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:48.315908 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.315908 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:48.315908 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:48.315908 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.315908 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:48.315908 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.315908 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:48.317404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:48.318153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:48.318153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:48.318153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.318153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:48.318153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:48.318153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.318153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:48.318153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.318153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.321537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:48.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:48.322310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:48.322531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:48.322904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:48.324389 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:48.324575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:48.324802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:48.325276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.325451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:48.325897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.327259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:48.327259 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.327259 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:48.327259 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:48.327259 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.327259 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:48.327259 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.327259 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:48.327812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.328059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:48.328319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.328319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:48.328939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:48.329311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:48.329822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:48.329822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:48.329822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:52.764496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:52.764496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:52.765092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:52.765611 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:52.767322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=239 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:02:52.831185 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:52.831452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:52.832340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:52.832340 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:52.832340 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:52.832340 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:52.832340 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:52.832340 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:52.832340 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:52.832340 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.068 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:52.833228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:52.833655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:52.834038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:52.834038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:52.834282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:52.834837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:52.835327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:52.835327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:52.835327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:53.285279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:53.285279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:53.285561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:53.286920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.287132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.287289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.287432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.288909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.289030 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.289183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.289330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.289915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-720d58e1-d208-4d00-adbd-63d377151d08, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.289915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-079b90c5-3c89-42c4-938b-d25f8ad92d80, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.289915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.290460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:53.290460 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.290460 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:53.290460 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:53.290460 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.290460 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:53.290460 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.290460 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:53.291538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.292016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:53.292354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.292354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:53.292943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:53.293747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.294715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:53.294715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.294715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:53.305538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:53.305873 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:53.306264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:53.306264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:53.306577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.306828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.308962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:53.308962 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.308962 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:53.308962 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:53.308962 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.308962 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:53.308962 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.308962 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:53.309934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:53.310267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:53.310267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:53.310267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.310267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:53.310267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:53.310267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.310267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:53.310267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.310267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.311831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:53.312221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:53.312530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.312530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:53.312530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.312530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:53.313423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:53.314215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:53.315232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.315232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:53.315232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:53.315232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.315232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:53.315232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.315232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:53.315232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:53.315232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:53.315232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:53.316143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:53.316143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.316143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:53.316302 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.316302 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:53.316302 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:53.316302 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.316302 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:53.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.316302 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:53.317173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:53.318115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:53.318115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:53.318115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.318115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:53.318115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:53.318115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.318115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:53.318115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.318115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.318639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:53.319099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:53.319862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.319862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:53.319862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.319862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.321697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:53.322477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:53.323018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.323018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:53.323018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.323018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:53.323931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:53.323931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:53.323931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:53.323931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:53.324231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.324547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:53.324927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.325724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:53.325724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.325724 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:53.325724 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:53.325724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.325724 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:53.325724 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.325724 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:53.326158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.326470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:53.326838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.326838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:53.327184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:53.327520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:53.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:53.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:53.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:57.764716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:57.765144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:57.765385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:57.766319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:57.802232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:57.812729 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.812933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.813349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:57.813349 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.813349 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:57.813349 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:57.813349 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.813349 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:57.813349 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.813349 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:57.814078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.814464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:57.814808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:57.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.815727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:57.815727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:57.815727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.815727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:57.815727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.815727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:57.815727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:57.815727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:57.822480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 27 16:02:57.822716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Number of inconsistencies found at create/update: networks=7, security_groups=2, routers=4, router_ports=4 {{(pid=92285) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 27 16:02:57.822870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Number of inconsistencies found at delete: router_ports=16 {{(pid=92285) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 27 16:02:57.822987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1a1659c0-4943-4985-b3f1-75f0c26af59f (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:57.865857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, resource={'id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'name': 'public', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['156facd3-ecc0-4e78-8b24-b1c2e5631a78', '3c538124-e9fa-4f89-905c-f13de628c91d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-27T15:57:46Z', 'updated_at': '2026-05-27T15:57:53Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.866777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.866777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1d74a028-7065-4269-8c77-c99e0ea79147, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.866777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.867498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:57.867498 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.867498 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:57.867498 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:57.867498 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.867498 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:57.867498 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.867498 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 1a1659c0-4943-4985-b3f1-75f0c26af59f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:57.870604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:57.871262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.871644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:57.871644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.871644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:57.871644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 23abdbc5-424f-449f-bf93-9dfba10b16d3 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:57.903170 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-23abdbc5-424f-449f-bf93-9dfba10b16d3, resource={'id': '23abdbc5-424f-449f-bf93-9dfba10b16d3', 'name': 'tempest-TestMultiAttachVolumeSwap-628050524-network', 'tenant_id': 'ed49473a6c7f4f49b7e4470332f047fd', 'project_id': 'ed49473a6c7f4f49b7e4470332f047fd', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['ab73621c-b37d-4b7d-8dfc-61a31822efe1'], 'standard_attr_id': 223, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:00:45Z', 'updated_at': '2026-05-27T16:00:46Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 62842}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.903453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-23abdbc5-424f-449f-bf93-9dfba10b16d3, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestMultiAttachVolumeSwap-628050524-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.903792 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-0bf92348-bf34-4fd7-9faf-64d9fa096459, external_ids_update=None, columns={'tag': 62842}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.903949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 23abdbc5-424f-449f-bf93-9dfba10b16d3 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:57.907038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:57.907527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:57.907892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:57.907892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:57.907892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.907892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:57.907892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:57.907892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.907892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:57.907892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.907892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:57.907892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 28588915-92a2-4ece-bd6f-c59cf1c236ce (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:57.937079 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28588915-92a2-4ece-bd6f-c59cf1c236ce, resource={'id': '28588915-92a2-4ece-bd6f-c59cf1c236ce', 'name': 'shared', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['73ef14d1-56af-457d-a4cc-cd982d0f7719'], 'standard_attr_id': 26, 'shared': True, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T15:58:53Z', 'updated_at': '2026-05-27T15:58:54Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 65356}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.937449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28588915-92a2-4ece-bd6f-c59cf1c236ce, col_values=(('external_ids', {'neutron:network_name': 'shared', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.937872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-854ee673-4c8a-4fe6-bb8f-35261288982d, external_ids_update=None, columns={'tag': 65356}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.938018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.938798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:57.938798 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.938798 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:57.938798 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:57.938798 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.938798 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:57.938798 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.938798 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 28588915-92a2-4ece-bd6f-c59cf1c236ce (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:57.939957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:57.940236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.940488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:57.940488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.940488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:57.940488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 517f9791-82bd-4c3f-a00e-708f7ab89356 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:57.969169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-517f9791-82bd-4c3f-a00e-708f7ab89356, resource={'id': '517f9791-82bd-4c3f-a00e-708f7ab89356', 'name': 'tempest-ServerActionsV293TestJSON-1661070470-network', 'tenant_id': 'ad60f81f688d408cb699813420f801b0', 'project_id': 'ad60f81f688d408cb699813420f801b0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['d6fa8a1d-5284-4721-bf01-cdc71a8350c4'], 'standard_attr_id': 77, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:00:20Z', 'updated_at': '2026-05-27T16:00:21Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 5776}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.969583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-517f9791-82bd-4c3f-a00e-708f7ab89356, col_values=(('external_ids', {'neutron:network_name': 'tempest-ServerActionsV293TestJSON-1661070470-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.969943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-087bf745-3889-4fa1-bbe0-54fb84691b04, external_ids_update=None, columns={'tag': 5776}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:57.970084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.971694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:57.971694 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.971694 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:57.971694 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:57.971694 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.971694 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:57.971694 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.971694 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 517f9791-82bd-4c3f-a00e-708f7ab89356 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:57.972360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:57.972643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:57.972854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:57.972854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:57.972854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:57.972854 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 5a1edaaa-4ed6-4d00-a56a-7791fadf7b43 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.001044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-5a1edaaa-4ed6-4d00-a56a-7791fadf7b43, resource={'id': '5a1edaaa-4ed6-4d00-a56a-7791fadf7b43', 'name': 'tempest-ServerActionsTestOtherB-1631689182-network', 'tenant_id': '16350f352afa48a196fcc51755d859c5', 'project_id': '16350f352afa48a196fcc51755d859c5', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['55dfd13d-fe82-43f9-9dc4-4b952a5be2a9'], 'standard_attr_id': 64, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:00:20Z', 'updated_at': '2026-05-27T16:00:21Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 11054}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.001248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-5a1edaaa-4ed6-4d00-a56a-7791fadf7b43, col_values=(('external_ids', {'neutron:network_name': 'tempest-ServerActionsTestOtherB-1631689182-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.001525 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b1cacf0-2727-4e38-9926-46fec6c12c72, external_ids_update=None, columns={'tag': 11054}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.001639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.002095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:58.002095 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.002095 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:58.002095 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:58.002095 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.002095 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:58.002095 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.002095 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 5a1edaaa-4ed6-4d00-a56a-7791fadf7b43 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.003195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:58.003420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.003682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:58.003682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.003682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:58.003682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 74879b31-a7b2-4ad0-ba36-2e67b54026bd (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.029685 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-74879b31-a7b2-4ad0-ba36-2e67b54026bd, resource={'id': '74879b31-a7b2-4ad0-ba36-2e67b54026bd', 'name': 'tempest-TestMultiAttachVolumeSwap-2050037902-network', 'tenant_id': '85d850ca7f5e4fb2bc021bb106a746bf', 'project_id': '85d850ca7f5e4fb2bc021bb106a746bf', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['7e818584-c1b6-4c45-a27f-2898fed8b2f6'], 'standard_attr_id': 171, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:00:34Z', 'updated_at': '2026-05-27T16:00:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 55262}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.029824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-74879b31-a7b2-4ad0-ba36-2e67b54026bd, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestMultiAttachVolumeSwap-2050037902-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.030037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d4e8f50d-8645-4a6c-a61c-ad6c1db58ef7, external_ids_update=None, columns={'tag': 55262}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.030133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.030696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:58.030696 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.030696 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:58.030696 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:58.030696 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.030696 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:58.030696 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.030696 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 74879b31-a7b2-4ad0-ba36-2e67b54026bd (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.031447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:58.031697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.031907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:58.031907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.031907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:58.031907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e2510e0f-28d1-484d-8605-6e6fbc1e01e2 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.063647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e2510e0f-28d1-484d-8605-6e6fbc1e01e2, resource={'id': 'e2510e0f-28d1-484d-8605-6e6fbc1e01e2', 'name': 'private', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2524c773-2910-41f1-b8dd-03d9ac809fd5', '71e0c14f-375e-43c9-bf49-7597bd311ee9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T15:57:42Z', 'updated_at': '2026-05-27T15:57:44Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 8519}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.063647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e2510e0f-28d1-484d-8605-6e6fbc1e01e2, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.063834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-96ddfe89-0ab3-4b19-b3ee-13a966e7d5c2, external_ids_update=None, columns={'tag': 8519}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.063834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.063834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:58.063834 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.063834 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:58.063834 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:58.063834 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.063834 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:58.063834 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.063834 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource e2510e0f-28d1-484d-8605-6e6fbc1e01e2 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.064604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:58.064835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.065122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:58.065122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.065122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:58.065122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 5370a7af-8582-437a-9649-803a7242ad2a (type: security_groups) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.116194 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Successfully bumped revision number for resource 5370a7af-8582-437a-9649-803a7242ad2a (type: security_groups) to 3
May 27 16:02:58.116194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource ea38b168-73ec-46e0-8cc1-cf43ddbd90a2 (type: security_groups) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.147563 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Successfully bumped revision number for resource ea38b168-73ec-46e0-8cc1-cf43ddbd90a2 (type: security_groups) to 3
May 27 16:02:58.147730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 17eeeecd-5fa9-44da-852d-69b27e40ad2a (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.245867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, resource={'id': '17eeeecd-5fa9-44da-852d-69b27e40ad2a', 'name': 'tempest-ServerActionsTestOtherB-1631689182-router', 'project_id': '16350f352afa48a196fcc51755d859c5', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.12'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::12d'}], 'enable_snat': True}, 'gw_port_id': '48f3f4fd-e6bd-4c52-9902-e1b594da530d', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.12', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::12d', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:00:22Z', 'updated_at': '2026-05-27T16:00:29Z', 'revision_number': 3, 'tenant_id': '16350f352afa48a196fcc51755d859c5'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.246275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, columns={'external_ids': {'neutron:router_name': 'tempest-ServerActionsTestOtherB-1631689182-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.246275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "48f3f4fd-e6bd-4c52-9902-e1b594da530d"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.246275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "48f3f4fd-e6bd-4c52-9902-e1b594da530d"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.246275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.246275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:58.246275 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.246275 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:58.246275 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:58.246590 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.246590 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:58.246590 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.246590 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:58.246590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 17eeeecd-5fa9-44da-852d-69b27e40ad2a. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 17eeeecd-5fa9-44da-852d-69b27e40ad2a (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:02:58.247868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:58.248194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:58.248578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:58.248578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:58.248578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.248578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:58.248578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:58.248578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.248578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:58.248578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.248578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:58.248578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 55c00c0e-f560-4d9a-ba79-16a405bd3265 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.271793 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Router 55c00c0e-f560-4d9a-ba79-16a405bd3265 does not have ID or gateway assigned {{(pid=92285) update_router /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/extensions/qos.py:527}}
May 27 16:02:58.272200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, resource={'id': '55c00c0e-f560-4d9a-ba79-16a405bd3265', 'name': 'tempest-TestMultiAttachVolumeSwap-2050037902-router', 'project_id': '85d850ca7f5e4fb2bc021bb106a746bf', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:00:37Z', 'updated_at': '2026-05-27T16:02:58Z', 'revision_number': 6, 'tenant_id': '85d850ca7f5e4fb2bc021bb106a746bf'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.272503 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DeleteLRouterExtGwCommand(_result=None, lrouter=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, if_exists=True, maintain_bfd=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.273020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): UpdateLRouterCommand(_result=None, name=neutron-55c00c0e-f560-4d9a-ba79-16a405bd3265, columns={'external_ids': {'neutron:router_name': 'tempest-TestMultiAttachVolumeSwap-2050037902-router', 'neutron:revision_number': '6', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.273302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.273886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:58.273886 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.273886 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:58.273886 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:58.273886 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.273886 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:58.273886 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.273886 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:58.274035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 55c00c0e-f560-4d9a-ba79-16a405bd3265. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 55c00c0e-f560-4d9a-ba79-16a405bd3265 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:02:58.275416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:58.275632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:58.275860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:58.275860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:58.275860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.275860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:58.275860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:58.275860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.275860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:58.275860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.275860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:58.275860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 77e3ef94-97b1-4fc8-8021-0a371dab8eff (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.285591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:58.306651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:58.315352 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:58.324403 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:02:58.362331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, resource={'id': '77e3ef94-97b1-4fc8-8021-0a371dab8eff', 'name': 'tempest-TestMultiAttachVolumeSwap-628050524-router', 'project_id': 'ed49473a6c7f4f49b7e4470332f047fd', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.29'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::129'}], 'enable_snat': True}, 'gw_port_id': '8306982f-e280-4988-aad3-3d0e363d80d3', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.29', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::129', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:00:47Z', 'updated_at': '2026-05-27T16:00:49Z', 'revision_number': 3, 'tenant_id': 'ed49473a6c7f4f49b7e4470332f047fd'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.362523 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, columns={'external_ids': {'neutron:router_name': 'tempest-TestMultiAttachVolumeSwap-628050524-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.362523 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "8306982f-e280-4988-aad3-3d0e363d80d3"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.362523 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "8306982f-e280-4988-aad3-3d0e363d80d3"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.362523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.362523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:58.362523 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.362523 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:58.362523 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:58.362913 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.362913 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:58.362913 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.362913 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:58.362913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 77e3ef94-97b1-4fc8-8021-0a371dab8eff. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 77e3ef94-97b1-4fc8-8021-0a371dab8eff (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:02:58.363723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:58.364020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:58.364368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:58.364368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:58.364368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.364368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:58.364368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:58.364368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.364368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:58.364368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.364368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:58.364368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource f7cc8645-aade-4001-9a99-2ecfa14c2cae (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.469420 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, resource={'id': 'f7cc8645-aade-4001-9a99-2ecfa14c2cae', 'name': 'tempest-ServerActionsV293TestJSON-1661070470-router', 'project_id': 'ad60f81f688d408cb699813420f801b0', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.218'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::151'}], 'enable_snat': True}, 'gw_port_id': '917bf073-0dcd-44b1-962e-ce1cf1407e9d', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.218', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::151', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:00:23Z', 'updated_at': '2026-05-27T16:00:29Z', 'revision_number': 3, 'tenant_id': 'ad60f81f688d408cb699813420f801b0'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.469693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, columns={'external_ids': {'neutron:router_name': 'tempest-ServerActionsV293TestJSON-1661070470-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.469693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "917bf073-0dcd-44b1-962e-ce1cf1407e9d"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.469693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "917bf073-0dcd-44b1-962e-ce1cf1407e9d"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.469693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.469693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:58.469693 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.469693 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:58.469693 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router f7cc8645-aade-4001-9a99-2ecfa14c2cae. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource f7cc8645-aade-4001-9a99-2ecfa14c2cae (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:58.469962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:58.470278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.470551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:58.470799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.470799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:58.470799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 06877545-428d-48dc-b0b8-938891fd89d1 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.497027 np0000001523 neutron-ovn-maintenance-worker[92285]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Skip fixing resource 06877545-428d-48dc-b0b8-938891fd89d1 (type: router_ports). Resource does not exist in Neutron database anymore: neutron_lib.exceptions.PortNotFound: Port 06877545-428d-48dc-b0b8-938891fd89d1 could not be found.
May 27 16:02:58.497286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 48f3f4fd-e6bd-4c52-9902-e1b594da530d (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.654267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-48f3f4fd-e6bd-4c52-9902-e1b594da530d, resource={'id': '48f3f4fd-e6bd-4c52-9902-e1b594da530d', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:22:6e:c2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '17eeeecd-5fa9-44da-852d-69b27e40ad2a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 112, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.12'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::12d'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:00:23Z', 'updated_at': '2026-05-27T16:00:25Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.654519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-48f3f4fd-e6bd-4c52-9902-e1b594da530d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.12/24', '2001:db8::12d/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.655307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=48f3f4fd-e6bd-4c52-9902-e1b594da530d, lrouter_port=lrp-48f3f4fd-e6bd-4c52-9902-e1b594da530d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.655307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.655542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:58.655542 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.655542 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:58.655542 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:58.655542 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.655542 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:58.655542 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.655542 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 48f3f4fd-e6bd-4c52-9902-e1b594da530d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.657002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:58.657266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.657602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:58.657602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.657602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:58.657602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 8306982f-e280-4988-aad3-3d0e363d80d3 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.810469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-8306982f-e280-4988-aad3-3d0e363d80d3, resource={'id': '8306982f-e280-4988-aad3-3d0e363d80d3', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:30:7a:63', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '77e3ef94-97b1-4fc8-8021-0a371dab8eff', 'device_owner': 'network:router_gateway', 'standard_attr_id': 235, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.29'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::129'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:00:47Z', 'updated_at': '2026-05-27T16:01:19Z', 'revision_number': 7, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.811059 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-8306982f-e280-4988-aad3-3d0e363d80d3, columns={'external_ids': {'neutron:revision_number': '7', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::129/64', '172.24.5.29/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.811059 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=8306982f-e280-4988-aad3-3d0e363d80d3, lrouter_port=lrp-8306982f-e280-4988-aad3-3d0e363d80d3, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.811059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.811059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:58.811059 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.811059 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:58.811059 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:58.811059 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.811059 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:58.811317 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.811317 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 8306982f-e280-4988-aad3-3d0e363d80d3 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.811651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:58.812160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.812576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:58.812576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.812576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:58.812576 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 917bf073-0dcd-44b1-962e-ce1cf1407e9d (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:02:58.926073 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-917bf073-0dcd-44b1-962e-ce1cf1407e9d, resource={'id': '917bf073-0dcd-44b1-962e-ce1cf1407e9d', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:6c:8c:6b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f7cc8645-aade-4001-9a99-2ecfa14c2cae', 'device_owner': 'network:router_gateway', 'standard_attr_id': 114, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.218'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::151'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:00:24Z', 'updated_at': '2026-05-27T16:00:27Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.926240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-917bf073-0dcd-44b1-962e-ce1cf1407e9d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::151/64', '172.24.5.218/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.926240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=917bf073-0dcd-44b1-962e-ce1cf1407e9d, lrouter_port=lrp-917bf073-0dcd-44b1-962e-ce1cf1407e9d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:58.926240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.926427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:58.926427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.926427 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:58.926427 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:58.926427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.926427 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:58.926427 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.926427 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 917bf073-0dcd-44b1-962e-ce1cf1407e9d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.927135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:58.927503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:58.927897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:58.927897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:58.927897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:58.927897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 06c01ddb-bcc5-4aac-9e55-df42ac0eedf7 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:58.927897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(06c01ddb-bcc5-4aac-9e55-df42ac0eedf7) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:58.935178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1faa0319-4124-458d-b85e-3f871546a68f (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:58.935481 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(1faa0319-4124-458d-b85e-3f871546a68f) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:58.941213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 2be78421-6686-4e64-8cf8-c04e3aa5b2c0 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:58.941543 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(2be78421-6686-4e64-8cf8-c04e3aa5b2c0) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:58.949202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 3159edef-b651-46bc-aee6-f89b389987f3 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:58.949461 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(3159edef-b651-46bc-aee6-f89b389987f3) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:58.954210 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 40900fd4-7a2f-4824-86ab-ab37867071e4 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:58.954452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(40900fd4-7a2f-4824-86ab-ab37867071e4) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:58.962087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 41131f12-62bb-4e0f-b3ab-ee5ebb0d321a (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:58.962341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(41131f12-62bb-4e0f-b3ab-ee5ebb0d321a) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:58.972787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 42197172-8882-49c6-80cc-01eb6d4be4bb (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:58.973023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(42197172-8882-49c6-80cc-01eb6d4be4bb) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:58.980061 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 4b2e65ee-2815-4f06-befa-bec2056664b6 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:58.980338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(4b2e65ee-2815-4f06-befa-bec2056664b6) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:58.989165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 545d35ab-38e9-440a-b5a7-82316316810f (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:58.989448 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(545d35ab-38e9-440a-b5a7-82316316810f) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:58.995806 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 5c823a43-b98d-4ae9-b897-65900ce3c211 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:58.996314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(5c823a43-b98d-4ae9-b897-65900ce3c211) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:59.002236 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 8285e23f-e019-47d6-9ccb-e2a5395491e6 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:59.002521 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(8285e23f-e019-47d6-9ccb-e2a5395491e6) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:59.007397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 921e76a6-390c-4e6b-b278-8346a3950177 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:59.007615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(921e76a6-390c-4e6b-b278-8346a3950177) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:59.019326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource a3427bc3-0c98-44af-bc17-f0552bd13440 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:59.021074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(a3427bc3-0c98-44af-bc17-f0552bd13440) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:59.030496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource b0263f8a-0912-4cb8-b2f0-0ef8601d13fa (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:59.030795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(b0263f8a-0912-4cb8-b2f0-0ef8601d13fa) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:59.039817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource c3ff8819-a499-4b1d-86ef-3b09ed703ec7 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:59.040214 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(c3ff8819-a499-4b1d-86ef-3b09ed703ec7) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:59.048313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource d365e9fb-d837-479c-9484-57c4f487ce32 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:02:59.048676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(d365e9fb-d837-479c-9484-57c4f487ce32) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:02:59.054869 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_for_inconsistencies finished in 1.240 seconds
May 27 16:02:59.055665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:59.055665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:59.056567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.056802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-77e3ef94-97b1-4fc8-8021-0a371dab8eff, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.056960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.057105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.057250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.057411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.057543 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.057703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.057860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.058036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.058615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:59.058615 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.058615 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:59.058615 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:59.058615 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.058615 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:59.058615 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.058615 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:59.059524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.059780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:59.059971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.059971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:59.060678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:59.061040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.061428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:59.061428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.061428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:59.061428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:59.061428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:59.061428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:02:59.061565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.061763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.063614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:59.064079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:59.064351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.064616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:59.064863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.064863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:59.064902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:59.065155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:59.065468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.065468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:59.065468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:59.065468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.065468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:59.065468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.065468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:59.065468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:59.065468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:59.065468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:59.066673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.066929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.067847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:59.067847 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.067847 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:59.067847 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:59.067847 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.067847 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:59.067847 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.067847 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:02:59.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:02:59.068598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:59.068598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:02:59.068598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.068598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:02:59.068598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:02:59.068598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.068598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:02:59.068598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.068598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.069125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:59.069320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:59.069554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.069554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:59.069554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.069554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.071134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:59.071682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:59.072082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.072082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:59.072082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.072082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:02:59.072082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:02:59.072082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:02:59.072082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:02:59.072593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.072865 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:02:59.073172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.073825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:02:59.073825 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.073825 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:02:59.073825 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:02:59.073825 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.073825 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:02:59.073825 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.073825 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:02:59.074948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.075215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:02:59.075505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:02:59.075821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:02:59.076119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:02:59.076119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:02:59.076119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:02:59.076119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:02:59.076119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:02:59.076119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:02.766942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:02.766942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:02.766942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:02.766942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:02.811732 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:02.811970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:02.814059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:02.814432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:02.814732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:02.814980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:02.815423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:02.815423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:02.815423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:02.815423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:02.815423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:02.815423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:02.815423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:02.815423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:02.815423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:04.055588 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:04.056285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:04.056357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:04.057758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.057933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.058056 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.058213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.058319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.058424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.058655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.058655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.058769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.058893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.059465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:04.059465 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.059465 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:04.059465 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:04.059465 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.059465 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:04.059465 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.059465 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:04.061068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.061441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:04.061761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:04.062140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.062569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:04.062569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:04.062569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.062569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:04.062569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.062569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:04.062569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:04.062569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:04.062569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:04.062569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:04.062921 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.062921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.062921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:04.062921 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.062921 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:04.062921 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:04.062921 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.062921 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:04.062921 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.062921 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:04.064733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:04.066484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:04.066948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.067272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:04.067754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:04.068204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:04.068855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.068855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:04.068855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:04.068855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.068855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:04.068855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.068855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:04.068855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:04.068855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:04.068855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:04.069097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:04.070611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:04.070917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:04.071241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:04.071528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.071727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.072019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:04.072302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:04.072302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:04.072302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:04.072302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.073695 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:04.074281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.074512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:04.074722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.074722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:04.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:04.076048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:04.076355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:04.076355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:04.076355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:07.767168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:07.767168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:07.767168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:07.767910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:07.835589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:07.836515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:07.836634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:07.836634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:07.836634 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:07.836634 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:07.836634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:07.836634 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:07.836634 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:07.836634 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.070 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:07.839648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:07.839915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:07.840185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:07.840508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:07.841184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:07.841184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:07.841184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:07.841184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:07.841184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:07.841184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:09.057969 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:09.058405 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:09.058773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:09.060123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.060294 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.060450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.060595 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.060757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.060908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.061052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.061192 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.061328 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.061462 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b647924-2dbd-4cec-b97c-62eeba0bc908, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.061609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.063245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:09.063757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:09.063757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.063757 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:09.063757 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:09.063757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.063757 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:09.063757 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.063757 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:09.065130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.065506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:09.065858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.065858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:09.066521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:09.067126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.067671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:09.067671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.067671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:09.067671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:09.067671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:09.067671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:09.068352 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:09.068414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.068694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.069451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:09.069451 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.069451 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:09.069451 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:09.069451 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.069451 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:09.069451 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.069451 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:09.070826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:09.071273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:09.071273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:09.071273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.071273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:09.071273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:09.071273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.071273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:09.071273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.071273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.072645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:09.073241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:09.073598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.073598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:09.073598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.073598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:09.300036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:09.300404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:09.300693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.300693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:09.300693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:09.300693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.300693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:09.300693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.300693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:09.302590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:09.303144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:09.303242 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:09.303879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:09.307371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.307530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.307967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:09.307967 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.307967 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:09.307967 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:09.307967 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.307967 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:09.307967 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.307967 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:09.308680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:09.309015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:09.309015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:09.309015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.309015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:09.309015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:09.309015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.309015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:09.309015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.309015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.309290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:09.309514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:09.309749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.309749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:09.309749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.309749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.310105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:09.310436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:09.311158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.311158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:09.311158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.311158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:09.311158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:09.311158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:09.311158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:09.311158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.311365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:09.311365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.311415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:09.311415 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.311415 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:09.311415 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:09.311415 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.311415 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:09.311415 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.311415 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:09.311992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.312234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:09.312477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.312477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:09.312703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:09.313038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:09.313433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:09.313433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:09.313433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:12.766885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:12.767509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:12.767509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:12.767951 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:12.829753 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:12.830083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:12.830586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:12.830586 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:12.830586 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:12.830586 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:12.830586 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:12.830586 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:12.830586 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:12.830586 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:12.831287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:12.831671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:12.832012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:12.832428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:12.833052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:12.833052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:12.833052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:12.833052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:12.833052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:12.833052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:14.059332 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:14.059910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:14.059910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:14.061965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.061965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.061965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.061965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.062172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.062172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.062330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.062473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.062661 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.062810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b647924-2dbd-4cec-b97c-62eeba0bc908, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.062943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-6b9054d2-9fbd-4bcf-95e8-aa78120430a5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.063075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.063716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:14.063716 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.063716 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:14.063716 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:14.063716 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.063716 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:14.063716 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.063716 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:14.064457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.064753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:14.064948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.064948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:14.066413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:14.067205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.067758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:14.067758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.067758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:14.067758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:14.067758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:14.067758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:14.067758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:14.067758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.068001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.071700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:14.072290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:14.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.073113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:14.073353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:14.073599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:14.074066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:14.074066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:14.074066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.074066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:14.074066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:14.074066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.074066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:14.074066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.074066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:14.303317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:14.303706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:14.303881 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:14.304056 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:14.305329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.305329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.305696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:14.305696 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.305696 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:14.305696 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:14.305696 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.305696 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:14.305696 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.305696 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:14.306269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:14.306702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:14.306702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:14.306702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.306702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:14.306702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:14.306702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.306702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:14.306702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.306702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.307134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:14.307447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:14.307948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.307948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:14.307948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.307948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.308212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:14.308865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:14.309372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.309372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:14.309372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.309372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:14.310360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:14.310546 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:14.310632 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:14.311121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:14.311599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.312116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:14.312684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.313434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:14.313434 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.313434 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:14.313434 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:14.313434 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.313434 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:14.313434 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.313434 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:14.314426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.314870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:14.315239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.315239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:14.315603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:14.316053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:14.316506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:14.316506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:14.316506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:17.767486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:17.767924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:17.767924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:17.768784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:17.825670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:17.825888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:17.826483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:17.826483 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:17.826483 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:17.826483 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:17.826483 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:17.826483 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:17.826483 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:17.826483 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.059 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:17.827510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:17.827895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:17.828208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:17.828208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:17.828740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:17.829300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:17.829713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:17.829713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:17.829713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:19.060373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:19.060875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:19.061014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:19.062367 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.062719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.062893 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.064789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.064789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.064789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.064789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.064789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.065188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.065188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b647924-2dbd-4cec-b97c-62eeba0bc908, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.065188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-6b9054d2-9fbd-4bcf-95e8-aa78120430a5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.065188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:19.067487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:19.067863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:19.068130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:19.068427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:19.068818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:19.068818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:19.068818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:19.068818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.068818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:19.068818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:19.068818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.068818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:19.068818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.068818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:19.069126 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:19.069613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:19.069874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:19.070120 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:19.070708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.070846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.073799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:19.074416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:19.075880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.077976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:19.078447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:19.078904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:19.079937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:19.079937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:19.079937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.079937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:19.079937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:19.079937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.079937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:19.079937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.079937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:19.303919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:19.306069 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:19.306069 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:19.306069 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:19.306069 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.306069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.306069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:19.306069 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.306069 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:19.306069 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:19.306508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.306902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.307178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:19.307362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:19.307583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.307583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:19.307583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.307583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.307954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:19.308345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:19.308754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.308754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:19.308754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.308754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:19.310696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:19.311664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:19.314645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:19.314645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:19.314645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.314645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:19.314645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.314645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:19.314645 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.314645 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:19.314645 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:19.314854 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.314854 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:19.314854 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.314854 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:19.315501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.315800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:19.316022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.316022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:19.316539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:19.316850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:19.317151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:19.317151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:19.317151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:22.769598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:22.770860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:22.770912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:22.771592 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:22.842897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:22.843137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:22.843899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:22.843899 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:22.843899 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:22.843899 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:22.843899 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:22.843899 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:22.843899 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:22.843899 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.073 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:22.845034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:22.845479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:22.845869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:22.845869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:22.846147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:22.850088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:22.850563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:22.850563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:22.850563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:24.063551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:24.063551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:24.063551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:24.064366 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.064517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.064638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.064721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.064803 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.064896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.064982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.065164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.065301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.065471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b647924-2dbd-4cec-b97c-62eeba0bc908, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.065618 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-6b9054d2-9fbd-4bcf-95e8-aa78120430a5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.065785 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ba9bb24-a0bc-4c25-ba10-ea1fe9cec220, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.065937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.067385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:24.067385 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.067385 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:24.067385 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:24.067385 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.067385 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:24.067385 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.067385 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:24.068549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.068812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:24.069110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.069110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:24.070789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:24.071220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.071734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:24.071734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.071734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:24.071734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:24.071734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:24.071734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:24.071734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:24.072395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.072694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.073136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:24.073136 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.073136 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:24.073136 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:24.073136 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.073136 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:24.073136 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.073136 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:24.073909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:24.074337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:24.074337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:24.074337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.074337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:24.074337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:24.074337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.074337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:24.074337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.074337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.075276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:24.075700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:24.076070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.076070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:24.076070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.076070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:24.078160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:24.078694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:24.079286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.079286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:24.079286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:24.079286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.079286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:24.079286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.079286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:24.304400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:24.305221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:24.305460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:24.305460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:24.305863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.306266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.308788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:24.308788 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.308788 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:24.308788 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:24.308788 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.308788 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:24.308788 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.308788 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:24.311824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:24.312494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:24.312494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:24.312494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.312494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:24.312494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:24.312494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.312494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:24.312494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.312494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:24.312494 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.313779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:24.314260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:24.314754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:24.316516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:24.316930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:24.317181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:24.317181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.317181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:24.317181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.317754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:24.317754 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.317754 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:24.317754 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:24.317754 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.317754 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:24.317754 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.317754 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:24.318776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.319043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:24.319727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:24.319982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:24.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:24.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:24.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:24.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:24.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:24.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:27.770900 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:27.771273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:27.771273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:27.772000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:27.810902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:27.811072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:27.811498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:27.811498 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:27.811498 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:27.811498 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:27.811498 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:27.811498 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:27.811498 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:27.811498 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:27.812239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:27.812555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:27.812782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:27.812782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:27.813013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:27.813876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:27.815641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:27.815641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:27.815641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:29.062479 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:29.062940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:29.063362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:29.064312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.064493 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.064709 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.064885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.065110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.065171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.065560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.065560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.065674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.066172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.066172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ba9bb24-a0bc-4c25-ba10-ea1fe9cec220, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.066172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.067076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:29.067076 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.067076 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:29.067076 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:29.067076 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.067076 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:29.067076 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.067076 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:29.067735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.068166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:29.068532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.068532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:29.069025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:29.069670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.070070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:29.070070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.070070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:29.071382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:29.071697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:29.071787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:29.071893 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:29.072184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.072443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.072736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:29.072736 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.072736 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:29.072736 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:29.072736 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.072736 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:29.072736 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.072736 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:29.073578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:29.073887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:29.073887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:29.073887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.073887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:29.073887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:29.073887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.073887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:29.073887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.073887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.074296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:29.074738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:29.075052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:29.075389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.075813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:29.308482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:29.308482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:29.308482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:29.308482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:29.308482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.308482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.308790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:29.308790 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.308790 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:29.308790 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:29.308790 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.308790 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:29.308790 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.308790 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:29.309602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:29.310061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:29.310061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:29.310061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.310061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:29.310061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:29.310061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.310061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:29.310061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.310061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.310596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:29.310948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:29.311349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.311349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:29.311349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.311349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.312755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:29.313270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:29.313835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.313835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:29.313835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.313835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:29.315078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:29.315591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:29.315777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:29.316061 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:29.316391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.317266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:29.317266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.317980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:29.317980 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.317980 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:29.317980 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:29.317980 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.317980 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:29.317980 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.317980 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:29.318944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.319216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:29.319490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.319490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:29.320007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:29.320300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:29.320752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:29.320752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:29.320752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:32.775035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:32.775353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:32.775581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:32.777702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:32.861525 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:32.861766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:32.863107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:32.863107 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:32.863107 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:32.863107 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:32.863107 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:32.863107 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:32.863107 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:32.863107 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.087 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:32.863542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:32.863818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:32.864096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:32.864096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:32.865871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:32.866332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:32.867100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:32.867100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:32.867100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:34.063053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:34.063486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:34.063534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:34.065000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.065213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.065396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.065560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.065797 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.065953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.066060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.066166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.066272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.066411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.066529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.067139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:34.067139 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.067139 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:34.067139 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:34.067139 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.067139 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:34.067139 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.067139 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:34.068041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.068319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:34.068554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.068554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:34.070954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:34.071437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.071919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:34.071919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.071919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:34.071919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:34.072463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:34.072575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:34.072730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:34.073056 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.073606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.074878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:34.074878 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.074878 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:34.074878 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:34.074878 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.074878 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:34.074878 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.074878 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:34.075563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:34.075909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:34.075909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:34.075909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.075909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:34.075909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:34.075909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.075909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:34.075909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.075909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.076811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:34.077277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:34.077773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.077773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:34.077773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.077773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:34.078237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:34.078791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:34.079264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.079264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:34.079264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:34.079264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.079264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:34.079264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.079264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:34.305829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:34.306181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:34.306355 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:34.306517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:34.308334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.308610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.310186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:34.310186 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.310186 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:34.310186 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:34.310186 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.310186 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:34.310186 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.310186 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:34.311226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:34.311638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:34.311638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:34.311638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.311638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:34.311638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:34.311638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.311638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:34.311638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.311638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.312263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:34.312559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:34.312964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.312964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:34.312964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.312964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.313418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:34.313909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:34.314465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.314465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:34.314465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.314465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:34.315618 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:34.317196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:34.317196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:34.317196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:34.317196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.317196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:34.318239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.318239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:34.318239 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.318239 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:34.318239 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:34.318239 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.318239 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:34.318239 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.318239 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:34.320167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.320524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:34.320927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.320927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:34.321284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:34.321957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:34.322592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:34.322592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:34.322592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:37.775372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:37.775853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:37.776124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:37.776652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:37.863987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:37.864196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:37.864707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:37.864707 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:37.864707 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:37.864707 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:37.864707 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:37.864707 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:37.864707 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:37.864707 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.089 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:37.865427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:37.865820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:37.866163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:37.866560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:37.867056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:37.867056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:37.867056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:37.867056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:37.867056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:37.867056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:39.064564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:39.064915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:39.065116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:39.066414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.066639 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.066786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.067362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.067362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.067362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.067362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.067362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.067675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.067733 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.067849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-17eeeecd-5fa9-44da-852d-69b27e40ad2a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.067945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.068694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:39.068694 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.068694 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:39.068694 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:39.068694 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.068694 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:39.068694 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.068694 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:39.069651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.070011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:39.070476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.070476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:39.070934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:39.071456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.072518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:39.072518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.072518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:39.072518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:39.072704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:39.072801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:39.072889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:39.073288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.073288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.074015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:39.074015 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.074015 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:39.074015 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:39.074015 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.074015 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:39.074015 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.074015 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:39.074535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:39.074946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:39.074946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:39.074946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.074946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:39.074946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:39.074946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.074946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:39.074946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.074946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.075193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:39.075550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:39.075922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:39.076323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.076848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:39.307221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:39.307221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:39.307393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:39.307572 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:39.307893 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.308078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.308709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:39.308709 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.308709 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:39.308709 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:39.308709 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.308709 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:39.308709 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.308709 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:39.309480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:39.310001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:39.310001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:39.310001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.310001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:39.310001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:39.310001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.310001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:39.310001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.310001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.310501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:39.310718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:39.310922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.310922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:39.310922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.310922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.312033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:39.312531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:39.312858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.312858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:39.312858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.312858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:39.317922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:39.317922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:39.317922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:39.317922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:39.317922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.317922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:39.318234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.318234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:39.318234 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.318234 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:39.318234 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:39.318234 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.318234 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:39.318234 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.318234 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:39.318806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.319066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:39.319272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.319272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:39.320142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:39.321138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:39.321659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:39.321659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:39.321659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:42.776370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:42.778967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:42.779111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:42.779790 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:42.831988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:42.832268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:42.833057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:42.833057 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:42.833057 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:42.833057 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:42.833057 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:42.833057 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:42.833057 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:42.833057 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:42.833879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:42.834413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:42.834810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:42.835297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:42.835972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:42.835972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:42.835972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:42.835972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:42.835972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:42.835972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:44.065105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:44.065557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:44.065710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:44.067163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.067423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.067577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.067722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.067837 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.067983 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.068108 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.068227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.068350 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.068463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-f7cc8645-aade-4001-9a99-2ecfa14c2cae, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.068564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.069907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:44.069907 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.069907 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:44.069907 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:44.069907 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.069907 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:44.069907 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.069907 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:44.071091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.071506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:44.071912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.071912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:44.072411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:44.072901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.073459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:44.073459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.073459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:44.073459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:44.073459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:44.073459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:44.073459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:44.073896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.074165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.074847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:44.074847 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.074847 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:44.074847 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:44.074847 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.074847 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:44.074847 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.074847 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:44.075753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:44.076060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:44.076060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:44.076060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.076060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:44.076060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:44.076060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.076060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:44.076060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.076060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:44.077091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:44.077485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.077485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:44.077485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.077485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:44.077866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:44.078348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:44.078967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.078967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:44.078967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:44.078967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.078967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:44.078967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.078967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:44.307336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:44.307810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:44.307998 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:44.308137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:44.308660 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.308911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.309462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:44.309462 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.309462 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:44.309462 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:44.309462 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.309462 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:44.309462 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.309462 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:44.310424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:44.310710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:44.310710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:44.310710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.310710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:44.310710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:44.310710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.310710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:44.310710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.310710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.311443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:44.311685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:44.311915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.311915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:44.311915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.311915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.313244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:44.313583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:44.313983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.313983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:44.313983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.313983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:44.316476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:44.317428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:44.317548 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:44.317740 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:44.318040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.318253 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:44.318576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.319047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:44.319047 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.319047 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:44.319047 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:44.319047 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.319047 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:44.319047 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.319047 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:44.319723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.319964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:44.320156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.320156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:44.320456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:44.320766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:44.321261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:44.321261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:44.321261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:47.779398 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:47.779939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:47.779939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:47.780813 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:47.831768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:47.831990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:47.832837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:47.832837 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:47.832837 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:47.832837 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:47.832837 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:47.832837 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:47.832837 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:47.832837 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:47.833596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:47.833896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:47.834164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:47.834164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:47.834734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:47.835321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:47.835766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:47.835766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:47.835766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:49.066717 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:49.069968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:49.070084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:49.071376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.071641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.071808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.072201 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.072201 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.072664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.072664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.073081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.073280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.073463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.073648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.074559 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:49.074994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:49.074994 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.074994 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:49.074994 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:49.074994 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.074994 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:49.074994 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.074994 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:49.076075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.076428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:49.076773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.076773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:49.077980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:49.079081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.079604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:49.079604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.079604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:49.079604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:49.079604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:49.081005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:49.081527 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.081900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.082515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:49.082515 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.082515 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:49.082515 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:49.082515 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.082515 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:49.082515 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.082515 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:49.083297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:49.084436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:49.084804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.085166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:49.085573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:49.086127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:49.086557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.086557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:49.086557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:49.086557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.086557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:49.086557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.086557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:49.308032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:49.308490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:49.308688 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:49.309026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:49.309381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.309613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.310447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:49.310447 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.310447 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:49.310447 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:49.310447 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.310447 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:49.310447 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.310447 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:49.311268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:49.311860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:49.311860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:49.311860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.311860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:49.311860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:49.311860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.311860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:49.311860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.311860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.312860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:49.313195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:49.313733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:49.314068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.314610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:49.317542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:49.317938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:49.318107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:49.318324 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:49.318822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.319077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:49.319332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.319917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:49.319917 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.319917 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:49.319917 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:49.319917 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.319917 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:49.319917 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.319917 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:49.321113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.321430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:49.321777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.321777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:49.323389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:49.323929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:49.324387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:49.324387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:49.324387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:52.780272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:52.780272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:52.780272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:52.781076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:52.782249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=762,neutron:UPDATE=2,neutron:DELETE=16 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:03:52.839963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:52.840195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:52.841008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:52.841008 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:52.841008 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:52.841008 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:52.841008 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:52.841008 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:52.841008 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:52.841008 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:52.842061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:52.843071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:52.843301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:52.843554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:52.843902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:52.843902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:52.843902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:52.843902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:52.843902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:52.843902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:54.071317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:54.071317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:54.071317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:54.072313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.072464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.072655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.072745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.073166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.073166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.073166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.073166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.073166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.073699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.073699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.073699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.074026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:54.074026 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.074026 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:54.074026 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:54.074026 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.074026 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:54.074026 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.074026 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:54.074820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.075168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:54.075481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:54.075907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.077191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:54.077191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:54.077191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.077191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:54.077191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.077191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:54.078026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:54.080099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:54.080215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:54.080293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:54.080913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.081176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.081894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:54.081894 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.081894 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:54.081894 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:54.081894 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.081894 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:54.081894 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.081894 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:54.082979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:54.083518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:54.083518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:54.083518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.083518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:54.083518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:54.083518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.083518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:54.083518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.083518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.084023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:54.084332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:54.084700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.084700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:54.084700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.084700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:54.086792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:54.087271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:54.087786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.087786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:54.087786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:54.087786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.087786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:54.087786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.087786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:54.308576 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:54.308961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:54.309490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:54.309490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:54.309764 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.310050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.310739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:54.310739 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.310739 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:54.310739 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:54.310739 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.310739 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:54.310739 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.310739 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:54.311806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:54.312340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:54.312340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:54.312340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.312340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:54.312340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:54.312340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.312340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:54.312340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.312340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.312864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:54.313189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:54.313594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.313594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:54.313594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.313594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.314031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:54.314386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:54.314734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.314734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:54.314734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.314734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:54.318133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:54.318459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:54.318648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:54.318844 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:54.320131 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.320513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:54.320895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.321589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:54.321589 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.321589 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:54.321589 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:54.321589 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.321589 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:54.321589 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.321589 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:54.322530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.323080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:54.323874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:54.324371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:54.324912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:54.324912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:54.324912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:54.324912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:54.324912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:54.324912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:57.780281 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:57.780838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:57.780838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:57.782208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:57.841250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:57.842159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:57.843802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:57.843802 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:57.843802 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:57.843802 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:57.843802 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:57.843802 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:57.843802 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:57.843802 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:57.845141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:57.845573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:57.845925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:57.845925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:57.846221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:57.846745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:57.847326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:57.847326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:57.847326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:59.071256 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:59.071807 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:59.071972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:59.073583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.073583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.073583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.073583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.073583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.073782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.073977 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.074033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.074154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.074291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.074414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.074582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.075297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:59.075297 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.075297 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:59.075297 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:59.075297 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.075297 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:59.075297 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.075297 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:59.076227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.076723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:59.076993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.076993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:59.077324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:59.078285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.078820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:59.078820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.078820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:59.080149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:59.080597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:59.080784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:59.080950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:03:59.081495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.081746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.082316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:59.082316 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.082316 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:59.082316 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:59.082316 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.082316 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:59.082316 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.082316 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:59.083356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:59.083753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:59.083753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:59.083753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.083753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:59.083753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:59.083753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.083753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:59.083753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.083753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.084382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:59.084923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:59.085228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.085228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:59.085228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.085228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:59.085543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:59.085848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:59.086182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.086182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:59.086182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:59.086182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.086182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:59.086182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.086182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:59.309912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:59.309912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:59.309912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:59.309912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:59.310833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.310833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.310833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:59.310833 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.310833 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:59.310833 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:59.310833 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.310833 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:59.310833 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.310833 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:03:59.312090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:03:59.312542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:59.312542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:03:59.312542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.312542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:03:59.312542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:03:59.312542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.312542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:03:59.312542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.312542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.312792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:59.313081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:59.313446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.313446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:59.313446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.313446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.314700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:59.315243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:59.315688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.315688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:59.315688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.315688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:03:59.318511 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:03:59.318869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:03:59.319125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:03:59.319240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:03:59.321225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.321225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:03:59.321225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.321395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:03:59.321395 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.321395 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:03:59.321395 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:03:59.321395 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.321395 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:03:59.321395 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.321395 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:03:59.322655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.323037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:03:59.323403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.323403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:03:59.324659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:03:59.325103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:03:59.325585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:03:59.325585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:03:59.325585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:02.780650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:02.781108 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:02.781168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:02.782111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:02.830972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:02.831219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:02.831897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:02.831897 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:02.831897 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:02.831897 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:02.831897 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:02.831897 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:02.831897 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:02.831897 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:02.832867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:02.833223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:02.833572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:02.833572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:02.834080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:02.834726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:02.835185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:02.835185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:02.835185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:04.072016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:04.072618 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:04.072618 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:04.073999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.073999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.073999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.073999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.074159 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.074159 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.074340 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.074488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.074646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.075054 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.075250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.076903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.076903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:04.076903 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.076903 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:04.076903 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:04.076903 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.076903 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:04.076903 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.076903 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:04.077528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.077899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:04.078211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.078211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:04.078965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:04.079533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.079987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:04.079987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.079987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:04.080593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:04.082741 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:04.082850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:04.082943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:04.084179 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.084395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.085005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:04.085005 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.085005 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:04.085005 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:04.085005 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.085005 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:04.085005 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.085005 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:04.085687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:04.086092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:04.086092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:04.086092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.086092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:04.086092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:04.086092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.086092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:04.086092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.086092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.086592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:04.087011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:04.087520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:04.087961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.088778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:04.310724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:04.311245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:04.311903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:04.311903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:04.312580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.312982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.314415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:04.314415 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.314415 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:04.314415 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:04.314415 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.314415 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:04.314415 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.314415 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:04.314787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:04.315234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:04.315586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.315819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.316802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:04.318840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:04.319179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.319179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:04.319179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.319179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:04.331831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:04.332878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:04.332878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:04.332878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:04.334603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.334953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:04.335651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.336400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:04.336400 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.336400 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:04.336400 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:04.336400 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.336400 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:04.336400 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.336400 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:04.337224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.338446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:04.338786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:04.339259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:04.339803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:04.339803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:04.339803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:04.339803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:04.339803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:04.339803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:07.781160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:07.781572 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:07.781655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:07.782539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:07.821264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:07.821448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:07.821946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:07.821946 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:07.821946 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:07.821946 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:07.821946 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:07.821946 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:07.821946 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:07.821946 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:07.822578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:07.822888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:07.823165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:07.823165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:07.823288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:07.823757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:07.824350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:07.824350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:07.824350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:09.072456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:09.073026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:09.073026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:09.075689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.075689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.075689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.075689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.075689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.075942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.075942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.078644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.078644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.078644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.078644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.078644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.079785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:09.079785 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.079785 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:09.079785 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:09.079785 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.079785 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:09.079785 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.079785 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:09.080639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.081102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:09.081454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:09.082130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.082643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:09.082643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:09.082643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.082643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:09.082643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.082643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:09.082824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:09.082980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:09.083055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:09.083154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:09.084372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.084372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.084542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:09.084542 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.084542 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:09.084542 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:09.084542 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.084542 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:09.084542 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.084542 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:09.085153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:09.085641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:09.085641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:09.085641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.085641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:09.085641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:09.085641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.085641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:09.085641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.085641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.085928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:09.086502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:09.086917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:09.087324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.087754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:09.311373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:09.311715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:09.311823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:09.312032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:09.312460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.312929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.313610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:09.313610 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.313610 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:09.313610 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:09.313610 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.313610 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:09.313610 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.313610 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:09.314263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:09.314739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:09.314739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:09.314739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.314739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:09.314739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:09.314739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.314739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:09.314739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.314739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.315034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:09.315336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:09.315740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:09.316134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.316693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:09.332686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:09.333093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:09.333179 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:09.333327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:09.334955 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.334955 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:09.334955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.335088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:09.335088 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.335088 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:09.335088 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:09.335088 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.335088 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:09.335088 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.335088 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:09.335991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.336359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:09.336774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.336774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:09.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:09.337351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:09.337975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:09.337975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:09.337975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:12.781706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:12.782272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:12.782431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:12.783359 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.065 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:12.848648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:12.849001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:12.849217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:12.849217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:12.849217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:12.849217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:12.849217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:12.849217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:12.849217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:12.849217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:12.849217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:12.849217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:12.850178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:12.850523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:12.850896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:12.850896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:12.850896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:14.074200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:14.075781 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:14.075781 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:14.076422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.076849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.077001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.077117 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.077231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.077345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.077506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.079782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.079782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.079782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.079782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.079782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.080494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:14.080494 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.080494 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:14.080494 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:14.080494 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.080494 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:14.080494 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.080494 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:14.084877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.085299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:14.085912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.085912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:14.086190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:14.087302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.088598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:14.088598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.088598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:14.088598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:14.088598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:14.088598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:14.088598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:14.088598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.088805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.088805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:14.088805 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.088805 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:14.088805 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:14.088805 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.088805 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:14.088805 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.088805 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:14.089029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:14.089495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:14.089495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:14.089495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.089495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:14.089495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:14.089495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.089495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:14.089495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.089495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.089958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:14.090309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:14.090670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.090670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:14.090670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.090670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:14.091105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:14.091798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:14.092245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.092245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:14.092245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:14.092245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.092245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:14.092245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.092245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:14.311750 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:14.312149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:14.312244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:14.312359 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:14.312807 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.312966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.313495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:14.313495 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.313495 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:14.313495 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:14.313495 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.313495 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:14.313495 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.313495 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:14.314137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:14.314444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:14.314444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:14.314444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.314444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:14.314444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:14.314444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.314444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:14.314444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.314444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.314826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:14.315113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:14.315333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.315333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:14.315333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.315333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.316009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:14.316320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:14.316606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.316606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:14.316606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.316606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:14.336173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:14.337189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:14.337189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:14.337189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:14.337438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.337645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:14.337980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.339824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:14.339824 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.339824 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:14.339824 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:14.339824 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.339824 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:14.339824 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.339824 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:14.340540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.340951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:14.341295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.341295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:14.342040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:14.342649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:14.343168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:14.343168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:14.343168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:17.783222 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:17.785955 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:17.786029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:17.787175 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:17.845237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:17.845732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:17.846309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:17.846309 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:17.846309 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:17.846309 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:17.846309 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:17.846309 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:17.846309 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:17.846309 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.060 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:17.847279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:17.848839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:17.849223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:17.849642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:17.850142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:17.850142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:17.850142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:17.850142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:17.850142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:17.850142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:19.075639 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:19.076869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:19.076869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:19.079642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.079751 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.080128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.080128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.080128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.080342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.080342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.080656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.080656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.080753 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.080831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.080950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.081560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:19.081560 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.081560 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:19.081560 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:19.081560 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.081560 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:19.081560 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.081560 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:19.082567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.082912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:19.083218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.083218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:19.083670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:19.087693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.088266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:19.088266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.088266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:19.088266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:19.088266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:19.088266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:19.088266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:19.093222 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.093222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.093222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:19.093222 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.093222 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:19.093222 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:19.093222 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.093222 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:19.093222 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.093222 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:19.097195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:19.097693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:19.098070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.100763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:19.101037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:19.101652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:19.102110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.102110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:19.102110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:19.102110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.102110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:19.102110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.102110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:19.313136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:19.314179 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:19.315700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:19.315700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:19.316878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.317679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.318790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:19.318790 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.318790 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:19.318790 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:19.318790 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.318790 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:19.318790 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.318790 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:19.321083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:19.322100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:19.322100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:19.322100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.322100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:19.322100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:19.322100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.322100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:19.322100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.322100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.328003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:19.328393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:19.328707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:19.329437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.330407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:19.336654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:19.337309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:19.337309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:19.337309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:19.339391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.339391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:19.339391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.339769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:19.339769 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.339769 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:19.339769 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:19.339769 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.339769 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:19.339769 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.339769 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:19.342831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.343128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:19.343371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:19.343699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:19.344060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:19.344060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:19.344060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:19.344060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:19.344060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:19.344060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:22.786258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:22.787310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:22.787310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:22.788003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:22.869796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:22.870773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.084 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:22.873728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:22.874392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:22.874716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:22.875124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:22.875822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:22.875822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:22.875822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:22.875822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:22.875822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:22.875822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:22.875822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:22.875822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:22.875822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:24.076730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:24.077270 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:24.077331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:24.078605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.078669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.081650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.081650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.081650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.081650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.081650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.081874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.081874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.081874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.081874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.081874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.081874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:24.081874 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.081874 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:24.081874 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.082045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:24.082346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:24.082697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:24.083031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:24.083394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:24.083394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:24.083394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.083394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:24.083394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:24.083394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.083394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:24.083394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.083394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:24.088864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:24.089277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:24.089389 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:24.089510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:24.089889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.090081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.090574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:24.090574 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.090574 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:24.090574 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:24.090574 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.090574 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:24.090574 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.090574 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:24.091377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:24.091748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:24.091748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:24.091748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.091748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:24.091748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:24.091748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.091748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:24.091748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.091748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.092289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:24.092596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:24.092947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.092947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:24.092947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.092947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:24.093333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:24.093733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:24.094183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.094183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:24.094183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:24.094183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.094183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:24.094183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.094183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:24.313782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:24.314796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:24.317838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:24.319460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:24.323346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.323346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.323872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:24.323872 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.323872 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:24.323872 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:24.323872 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.323872 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:24.323872 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.323872 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:24.324879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:24.325275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:24.325275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:24.325275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.325275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:24.325275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:24.325275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.325275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:24.325275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.325275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.326863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:24.327265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:24.327726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.327726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:24.327726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.327726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.328275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:24.328773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:24.329307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.329307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:24.329307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.329307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:24.337012 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:24.337397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:24.337523 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:24.337732 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:24.338270 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.338764 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:24.339267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.340142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:24.340142 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.340142 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:24.340142 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:24.340142 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.340142 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:24.340142 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.340142 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:24.341414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.341858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:24.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:24.343026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:24.343672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:24.343672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:24.343672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:24.343672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:24.343672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:24.343672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:27.787110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:27.787692 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:27.787692 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:27.788674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:27.842250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:27.842595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:27.843407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:27.843407 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:27.843407 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:27.843407 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:27.843407 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:27.843407 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:27.843407 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:27.843407 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.056 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:27.844603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:27.844835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:27.845115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:27.845115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:27.846566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:27.846871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:27.847181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:27.847181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:27.847181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:29.077249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:29.077867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:29.077867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:29.079333 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.079524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.079706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.080096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.080330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.080607 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.080830 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.081051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.081204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.081345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.081638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.081942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.082866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:29.082866 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.082866 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:29.082866 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:29.082866 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.082866 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:29.082866 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.082866 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:29.084289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.084613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:29.084911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.084911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:29.087048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:29.087539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.088071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:29.088071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.088071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:29.089212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:29.089519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:29.089853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:29.090028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:29.090731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.091070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.091702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:29.091702 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.091702 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:29.091702 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:29.091702 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.091702 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:29.091702 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.091702 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:29.092534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:29.092944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:29.092944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:29.092944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.092944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:29.092944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:29.092944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.092944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:29.092944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.092944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.093210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:29.093480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:29.093750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.093750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:29.093750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.093750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:29.094590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:29.094928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:29.095194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.095194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:29.095194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:29.095194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.095194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:29.095194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.095194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:29.314597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:29.315076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:29.315244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:29.315407 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:29.315959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.316376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.317096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:29.317096 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.317096 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:29.317096 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:29.317096 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.317096 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:29.317096 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.317096 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:29.318186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:29.318587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:29.318587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:29.318587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.318587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:29.318587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:29.318587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.318587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:29.318587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.318587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.319770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:29.320101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:29.320499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:29.320871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.321421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:29.337934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:29.338062 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:29.338221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:29.338434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:29.339287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.339918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:29.340550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.341135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:29.341135 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.341135 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:29.341135 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:29.341135 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.341135 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:29.341135 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.341135 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:29.341925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.342283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:29.342602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.342602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:29.343318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:29.343898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:29.344377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:29.344377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:29.344377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:32.787648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:32.788127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:32.788451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:32.789255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:32.836785 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:32.838048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:32.838840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:32.838840 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:32.838840 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:32.838840 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:32.838840 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:32.838840 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:32.838840 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:32.838840 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:32.839612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:32.839902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:32.840108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:32.840108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:32.840404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:32.840711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:32.841103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:32.841103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:32.841103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:34.078194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:34.083843 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:34.083843 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:34.084581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.084809 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.085020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.085322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.085322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.085479 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.085541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.085753 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.085851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.085915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.086054 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.086481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.089973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:34.089973 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.089973 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:34.089973 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:34.089973 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.089973 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:34.089973 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.089973 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:34.090969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.091426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:34.092284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.092284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:34.094059 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:34.096581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:34.097074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.097562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:34.097562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.097562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:34.097562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:34.097562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:34.097562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:34.097940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.098147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.098538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:34.098538 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.098538 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:34.098538 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:34.098538 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.098538 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:34.098538 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.098538 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:34.099342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:34.099804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:34.099804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:34.099804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.099804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:34.099804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:34.099804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.099804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:34.099804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.099804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.100243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:34.100643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:34.101024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.101024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:34.101024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.101024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:34.102189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:34.102614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:34.103090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.103090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:34.103090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:34.103090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.103090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:34.103090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.103090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:34.315114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:34.315509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:34.315704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:34.315909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:34.316319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.316548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.317098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:34.317098 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.317098 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:34.317098 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:34.317098 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.317098 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:34.317098 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.317098 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:34.318077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:34.318350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:34.318350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:34.318350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.318350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:34.318350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:34.318350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.318350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:34.318350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.318350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.318964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:34.319184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:34.319427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.319427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:34.319427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.319427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.319987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:34.320303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:34.320584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.320584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:34.320584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.320584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:34.338179 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:34.338443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:34.338562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:34.338794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:34.339108 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.339364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:34.339725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.340242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:34.340242 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.340242 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:34.340242 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:34.340242 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.340242 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:34.340242 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.340242 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:34.341249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.341474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:34.341721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.341721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:34.342564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:34.342911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:34.343195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:34.343195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:34.343195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:37.788292 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:37.788900 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:37.788988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:37.790641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:37.837479 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:37.837753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:37.838345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:37.838345 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:37.838345 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:37.838345 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:37.838345 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:37.838345 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:37.838345 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:37.838345 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:37.839312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:37.839573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:37.839836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:37.839836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:37.840451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:37.840949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:37.841323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:37.841323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:37.841323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:39.080338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:39.081960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:39.081960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:39.082909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.083114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.083278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.083431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.083600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.083777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.083928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.084285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.084556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.084861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.085021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.085301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:39.088323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:39.088866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:39.089101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:39.089434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:39.090871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:39.090871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:39.090871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:39.090871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.090871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:39.090871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:39.090871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.090871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:39.090871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.090871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:39.097062 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:39.097683 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:39.097887 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:39.098116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:39.098731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.098943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.099885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:39.099885 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.099885 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:39.099885 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:39.099885 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.099885 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:39.099885 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.099885 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:39.102900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:39.103890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:39.104524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.104929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:39.106714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:39.107585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:39.108532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.108532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:39.108532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:39.108532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.108532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:39.108532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.108532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:39.315636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:39.316412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:39.316412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:39.316412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:39.316730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.316990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.319773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:39.320109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:39.320352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.320557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:39.320782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:39.321015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:39.321316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:39.321316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.321316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:39.321316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:39.321316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.321316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:39.321316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.321316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:39.338577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:39.339104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:39.339383 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:39.339644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:39.340235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.340581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:39.340976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:39.343213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:39.343824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:39.344951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:39.345556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:39.346490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:39.346490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:39.346490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:39.346490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:39.346490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:39.346490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:39.346490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:39.346490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:39.346490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:39.346490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:42.789831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:42.789831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:42.789831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:42.790635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:42.838860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:42.838918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:42.840280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:42.840280 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:42.840280 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:42.840280 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:42.840280 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:42.840280 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:42.840280 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:42.840280 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:42.840593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:42.840944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:42.841305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:42.841305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:42.842485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:42.843781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:42.844194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:42.844194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:42.844194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:44.081258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:44.085705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:44.085705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:44.086138 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.086138 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.086335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.086431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.086540 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.086665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.086774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.086875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.086993 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.090770 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:44.094762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.095085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:44.095368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.095368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:44.096009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:44.096390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.096856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:44.096856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.096856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:44.097641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:44.097885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:44.098005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:44.099663 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:44.100293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.100293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.100722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:44.100722 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.100722 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:44.100722 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:44.100722 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.100722 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:44.100722 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.100722 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:44.102753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:44.103275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:44.103652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.103997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:44.104288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:44.104708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:44.105143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.105143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:44.105143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:44.105143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.105143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:44.105143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.105143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:44.316088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:44.317405 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:44.318110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:44.318250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:44.319755 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.321484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.321590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:44.321590 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.321590 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:44.321590 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:44.321590 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.321590 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:44.321590 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.321590 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:44.323852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:44.324351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:44.324351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:44.324351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.324351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:44.324351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:44.324351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.324351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:44.324351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.324351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:44.326945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:44.327302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.327302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:44.327302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.327302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.327896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:44.328277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:44.328750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.328750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:44.328750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.328750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:44.339244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:44.340566 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:44.341867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:44.343152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:44.345032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.347257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:44.348230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.351085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:44.351085 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.351085 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:44.351085 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:44.351085 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.351085 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:44.351085 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.351085 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:44.359307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.359813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:44.360374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.360374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:44.362284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:44.362748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:44.363217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:44.363217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:44.363217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:47.789686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:47.791364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:47.791364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:47.791364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:47.846877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:47.847197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:47.847879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:47.847879 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:47.847879 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:47.847879 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:47.847879 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:47.847879 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:47.847879 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:47.847879 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.058 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:47.848971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:47.849552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:47.850311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:47.850895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:47.851381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:47.851381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:47.851381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:47.851381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:47.851381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:47.851381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:49.084158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:49.084721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:49.084721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:49.089078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.089249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.089651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.089651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.089651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.089777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.089883 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.090033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.090195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.090335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.090396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.090503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.091021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:49.091021 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.091021 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:49.091021 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:49.091021 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.091021 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:49.091021 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.091021 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:49.091968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.092390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:49.093201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:49.093683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.094339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:49.094339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:49.094339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.094339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:49.094339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.094339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:49.098154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:49.098596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:49.098796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:49.098869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:49.099379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.100037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.101753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:49.101753 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.101753 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:49.101753 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:49.101753 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.101753 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:49.101753 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.101753 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:49.102602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:49.103123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:49.103123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:49.103123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.103123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:49.103123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:49.103123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.103123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:49.103123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.103123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.103598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:49.103987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:49.104364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.104364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:49.104364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.104364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:49.105250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:49.105556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:49.106007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.106007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:49.106007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:49.106007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.106007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:49.106007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.106007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:49.317040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:49.317668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:49.317668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:49.317668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:49.317868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.318102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.320641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:49.321020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:49.321266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.321483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:49.321744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:49.322039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:49.322382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:49.322382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.322382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:49.322382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:49.322382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.322382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:49.322382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.322382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:49.343653 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:49.343653 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:49.343653 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:49.343653 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:49.343653 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.343653 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.343983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:49.344428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:49.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:49.344922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:49.345231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:52.790258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:52.790795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:52.790795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:52.791375 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:52.805408 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=241 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:04:52.857820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:52.857991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:52.858692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:52.858692 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:52.858692 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:52.858692 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:52.858692 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:52.858692 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:52.858692 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:52.858692 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.068 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:52.859974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:52.860379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:52.860810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:52.860810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:52.861189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:52.861712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:52.862342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:52.862342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:52.862342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:54.084597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:54.085132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:54.085132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:54.086079 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.086253 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.086395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.086505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.086613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.086763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.086886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.086994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.087091 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.087187 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.087278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.087407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.087834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:54.087834 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.087834 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:54.087834 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:54.087834 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.087834 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:54.087834 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.087834 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:54.088513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.088809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:54.089031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.089031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:54.089309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:54.089582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.089984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:54.089984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.089984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:54.098578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:54.098866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:54.098979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:54.099072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:54.099417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.099638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.100119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:54.100119 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.100119 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:54.100119 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:54.100119 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.100119 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:54.100119 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.100119 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:54.100748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:54.101163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:54.101163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:54.101163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.101163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:54.101163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:54.101163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.101163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:54.101163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.101163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.101407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:54.101763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:54.102131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.102131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:54.102131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.102131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:54.102243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:54.102683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:54.103136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.103136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:54.103136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:54.103136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.103136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:54.103136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.103136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:54.317383 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:54.317838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:54.317980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:54.318060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:54.318499 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.318835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.319313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:54.319313 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.319313 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:54.319313 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:54.319313 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.319313 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:54.319313 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.319313 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:54.320016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:54.320946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:54.321758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:54.322191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:54.322497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.323002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:54.340682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:54.341096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:54.341537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:54.341537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:54.342119 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.342393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:54.342783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.343413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:54.343413 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.343413 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:54.343413 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:54.343413 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.343413 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:54.343413 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.343413 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:54.344442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.345095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:54.345392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:54.345818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:54.346234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:54.346234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:54.346234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:54.346234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:54.346234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:54.346234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:57.790680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:57.791177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:57.791177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:57.792022 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:57.829835 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:57.830013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:57.830562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:57.830562 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:57.830562 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:57.830562 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:57.830562 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:57.830562 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:57.830562 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:57.830562 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:57.831372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:57.831726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:57.832064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:57.832064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:57.832134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:57.832389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:57.832749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:57.832749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:57.832749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:59.085340 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:59.085816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:59.085918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:59.087192 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.087334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.087415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.087557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.087774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.087824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.087947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.088002 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.088161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.088495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.088495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.088495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.089105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:59.089105 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.089105 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:59.089105 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:59.089105 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.089105 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:59.089105 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.089105 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:59.089938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.090977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:59.091596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:59.092087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.092713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:59.092713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:59.092713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.092713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:59.092713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.092713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:59.099976 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:59.100321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:59.100386 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:59.100511 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:04:59.101065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.101270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.101817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:59.101817 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.101817 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:59.101817 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:59.101817 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.101817 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:59.101817 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.101817 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:59.102586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:59.103103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:59.103103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:59.103103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.103103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:59.103103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:59.103103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.103103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:59.103103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.103103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.103386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:59.103788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:59.104320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:59.104782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.105330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:59.317914 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:59.318302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:59.318302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:59.318460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:59.319124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.319433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.320221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:59.320221 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.320221 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:59.320221 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:59.320221 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.320221 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:59.320221 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.320221 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:04:59.320935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:59.321707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:59.322063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:59.322370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:04:59.322886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.323394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:04:59.341209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:04:59.341637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:04:59.341806 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:04:59.342061 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:04:59.342572 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.342869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:04:59.343270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.343892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:04:59.343892 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.343892 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:04:59.343892 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:04:59.343892 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.343892 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:04:59.343892 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.343892 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:04:59.344636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.345257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:04:59.345576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:04:59.345964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:04:59.346394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:04:59.346394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:04:59.346394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:04:59.346394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:04:59.346394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:04:59.346394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:02.791137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:02.791673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:02.791673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:02.792345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:02.834041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:02.834223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:02.834674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:02.834674 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:02.834674 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:02.834674 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:02.834674 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:02.834674 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:02.834674 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:02.834674 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:02.835418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:02.835865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:02.836310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:02.836767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:02.837319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:02.837319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:02.837319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:02.837319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:02.837319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:02.837319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:04.086925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:04.087533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:04.087690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:04.089116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.089432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.089590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.089816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.090064 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.090251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.090406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.090580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.090745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.090902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.091076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.091271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.092986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:04.092986 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.092986 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:04.092986 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:04.092986 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.092986 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:04.092986 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.092986 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:04.094967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.095376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:04.095760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.095760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:04.097101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:04.097571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.098055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:04.098055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.098055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:04.100843 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:04.102201 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:04.102332 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:04.102474 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:04.105140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.105424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.106311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:04.106311 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.106311 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:04.106311 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:04.106311 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.106311 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:04.106311 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.106311 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:04.107190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:04.107583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:04.107583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:04.107583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.107583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:04.107583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:04.107583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.107583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:04.107583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.107583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.108122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:04.108691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:04.109089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.109089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:04.109089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.109089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:04.110005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:04.110501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:04.110979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.110979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:04.110979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:04.110979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.110979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:04.110979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.110979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:04.318303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:04.318633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:04.318866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:04.319064 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:04.320043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.320236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.320902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:04.320902 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.320902 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:04.320902 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:04.320902 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.320902 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:04.320902 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.320902 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:04.322105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:04.322658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:04.322658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:04.322658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.322658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:04.322658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:04.322658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.322658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:04.322658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.322658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.324353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:04.324704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:04.325056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.325056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:04.325056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.325056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.327836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:04.328437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:04.328910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.328910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:04.328910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.328910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:04.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:04.342759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:04.342759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:04.342759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:04.343269 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.343558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:04.344077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.346723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:04.346723 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.346723 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:04.346723 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:04.346723 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.346723 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:04.346723 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.346723 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:04.350176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.350749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:04.351108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:04.351650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:04.352271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:04.352271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:04.352271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:04.352271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:04.352271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:04.352271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:07.791889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:07.792370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:07.792613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:07.793575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:07.859265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:07.859467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:07.860818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:07.860818 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:07.860818 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:07.860818 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:07.860818 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:07.860818 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:07.860818 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:07.860818 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.068 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:07.861285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:07.861583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:07.861926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:07.862497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:07.862965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:07.862965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:07.862965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:07.862965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:07.862965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:07.862965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:09.087428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:09.088293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:09.088398 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:09.089951 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.090172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.090350 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.090505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.090617 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.090877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.091035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.091205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.091357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.091516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.091866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.092070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.092605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:09.092605 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.092605 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:09.092605 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:09.092605 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.092605 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:09.092605 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.092605 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:09.093516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.093897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:09.094958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:09.095696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.096142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:09.096142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:09.096142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.096142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:09.096142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.096142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:09.102163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:09.102459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:09.102564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:09.102706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:09.103139 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.103422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.104047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:09.104047 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.104047 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:09.104047 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:09.104047 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.104047 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:09.104047 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.104047 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:09.104742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:09.105031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:09.105031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:09.105031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.105031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:09.105031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:09.105031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.105031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:09.105031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.105031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.105669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:09.106143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:09.106381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.106381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:09.106381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.106381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:09.107066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:09.107502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:09.107823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.107823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:09.107823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:09.107823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.107823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:09.107823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.107823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:09.318787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:09.319969 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:09.320421 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:09.320421 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:09.321932 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.322254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.323254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:09.323254 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.323254 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:09.323254 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:09.323254 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.323254 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:09.323254 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.323254 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:09.324808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:09.325266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:09.325266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:09.325266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.325266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:09.325266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:09.325266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.325266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:09.325266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.325266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.325521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:09.325836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:09.326170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.326170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:09.326170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.326170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.326361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:09.326821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:09.327274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.327274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:09.327274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.327274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:09.342676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:09.342676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:09.343325 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:09.343325 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:09.345596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.345596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:09.345923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.346789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:09.346789 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.346789 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:09.346789 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:09.346789 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.346789 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:09.346789 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.346789 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:09.348479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.348937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:09.349513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:09.351191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:09.353478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:09.353478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:09.353478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:09.353478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:09.353478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:09.353478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:12.792343 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:12.792908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:12.792908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:12.796666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:12.849501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:12.850120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:12.851039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:12.851039 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:12.851039 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:12.851039 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:12.851039 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:12.851039 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:12.851039 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:12.851039 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.058 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:12.851859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:12.852833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:12.853177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:12.853564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:12.854064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:12.854064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:12.854064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:12.854064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:12.854064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:12.854064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:14.088393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:14.088954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:14.088954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:14.090282 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.090591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.090757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.090892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.091084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.091261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.091400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.091529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.091712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.091858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.092003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.092149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.093871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:14.093871 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.093871 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:14.093871 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:14.093871 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.093871 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:14.093871 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.093871 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:14.094874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.095289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:14.095718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.095718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:14.096053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:14.096673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.097275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:14.097275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.097275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:14.102678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:14.102971 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:14.103141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:14.103141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:14.103605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.104161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.104161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:14.104161 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.104161 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:14.104161 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:14.104161 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.104161 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:14.104161 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.104161 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:14.104752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:14.105504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:14.106978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.107382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:14.107736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:14.108156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:14.109115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.109115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:14.109115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:14.109115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.109115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:14.109115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.109115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:14.320040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:14.321906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:14.322022 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:14.322140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:14.322577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.322744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.323139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:14.323139 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.323139 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:14.323139 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:14.323139 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.323139 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:14.323139 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.323139 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:14.323738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:14.324014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:14.324014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:14.324014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.324014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:14.324014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:14.324014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.324014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:14.324014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.324014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.324396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:14.324702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:14.324966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.324966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:14.324966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.324966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.325153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:14.325564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:14.325910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.325910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:14.325910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.325910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:14.343085 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:14.343506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:14.343674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:14.343897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:14.344204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.344452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:14.344811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.346909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:14.346909 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.346909 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:14.346909 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:14.346909 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.346909 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:14.346909 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.346909 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:14.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.347922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:14.348158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.348158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:14.348401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:14.349097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:14.349499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:14.349499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:14.349499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:17.793841 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:17.799168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:17.799302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:17.800058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:17.859318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:17.859672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:17.861980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:17.861980 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:17.861980 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:17.861980 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:17.861980 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:17.861980 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:17.861980 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:17.861980 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:17.862825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:17.864043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:17.865339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:17.865971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:17.867255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:17.867255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:17.867255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:17.867255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:17.867255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:17.867255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:19.089209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:19.089756 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:19.089756 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:19.091117 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.091555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.092191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.092557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.092769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.093153 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.093390 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.093568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.093876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.094154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.094327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.094482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.096871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:19.096871 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.096871 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:19.096871 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:19.096871 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.096871 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:19.096871 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.096871 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:19.097607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.097965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:19.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:19.098426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:19.098963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.100371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:19.100371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.100371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:19.103356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:19.104071 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:19.104239 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:19.104416 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:19.104928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.105149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.105700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:19.105700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.105700 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:19.105700 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:19.105700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.105700 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:19.105700 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.105700 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:19.106692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:19.107408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:19.107408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:19.107408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.107408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:19.107408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:19.107408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.107408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:19.107408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.107408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.108163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:19.108650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:19.111307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:19.113703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.114424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:19.322234 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:19.322502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:19.322571 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:19.322720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:19.325946 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.328756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.329503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:19.329503 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.329503 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:19.329503 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:19.329503 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.329503 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:19.329503 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.329503 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:19.331040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:19.331696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:19.332181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:19.332563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:19.333360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.333917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:19.343747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:19.344013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:19.344192 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:19.344402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:19.344839 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.345055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:19.345419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.345945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:19.345945 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.345945 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:19.345945 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:19.345945 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.345945 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:19.345945 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.345945 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:19.347390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.347852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:19.348219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.348219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:19.348431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:19.348891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:19.349407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:19.349407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:19.349407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:22.799267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:22.799811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:22.800046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:22.801111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:22.861302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:22.861800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:22.863185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:22.863185 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:22.863185 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:22.863185 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:22.863185 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:22.863185 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:22.863185 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:22.863185 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.062 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:22.863499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:22.863869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:22.864342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:22.864342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:22.864561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:22.865003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:22.865492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:22.865492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:22.865492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:24.089697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:24.090086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:24.090086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:24.092333 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.092464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.092582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.092731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.094251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.094251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.094251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.094251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.094251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.094534 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:24.095767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.096263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:24.096653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.096653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:24.096833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:24.097351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.097878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:24.097878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.097878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:24.104083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:24.104357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:24.104457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:24.104577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:24.105192 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.105410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.105948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:24.105948 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.105948 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:24.105948 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:24.105948 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.105948 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:24.105948 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.105948 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:24.106864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:24.107465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:24.107465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:24.107465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.107465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:24.107465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:24.107465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.107465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:24.107465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.107465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.107983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:24.108191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:24.108410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.108410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:24.108410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.108410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:24.108838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:24.109181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:24.109456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.109456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:24.109456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:24.109456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.109456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:24.109456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.109456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:24.322534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:24.322977 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:24.322977 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:24.323183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:24.323570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.324543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.324543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:24.324543 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.324543 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:24.324543 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:24.324543 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.324543 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:24.324543 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.324543 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:24.325788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:24.326191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:24.326598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.326948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.328234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:24.329064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:24.329713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.329713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:24.329713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.329713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:24.344669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:24.345111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:24.345351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:24.345793 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.349731 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:24.350251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.350568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:24.350927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.350927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:24.351487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:24.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:24.352562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:24.352562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:24.352562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:27.804689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:27.804689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:27.804689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:27.804689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:27.851286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:27.851530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:27.852021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:27.852021 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:27.852021 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:27.852021 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:27.852021 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:27.852021 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:27.852021 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:27.852021 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:27.852721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:27.852983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:27.853186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:27.853186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:27.853478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:27.853816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:27.854232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:27.854232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:27.854232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:29.090171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:29.090525 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:29.090667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:29.091870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.092600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.092600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.092600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.092600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.092600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.092936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.092936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.092936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.093035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-23b73947-a4c3-4fca-98e1-6d0ef19bcb78, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.093194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.093256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.093700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:29.093700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.093700 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:29.093700 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:29.093700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.093700 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:29.093700 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.093700 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:29.094643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.095003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:29.095369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.095369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:29.095592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:29.096058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.096606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:29.096606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.096606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:29.104664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:29.104937 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:29.105058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:29.105199 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:29.105641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.105911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.106552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:29.106552 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.106552 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:29.106552 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:29.106552 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.106552 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:29.106552 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.106552 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:29.107506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:29.107908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:29.107908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:29.107908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.107908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:29.107908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:29.107908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.107908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:29.107908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.107908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.108753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:29.109063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:29.109519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.109519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:29.109519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.109519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:29.109654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:29.110051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:29.110639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.110639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:29.110639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:29.110639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.110639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:29.110639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.110639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:29.322958 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:29.323297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:29.323438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:29.323640 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:29.323994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.325746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.325746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:29.325746 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.325746 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:29.325746 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:29.325746 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.325746 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:29.325746 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.325746 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:29.326058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:29.326434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:29.326434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:29.326434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.326434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:29.326434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:29.326434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.326434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:29.326434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.326434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.326877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:29.327183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:29.327531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.327531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:29.327531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.327531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.328041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:29.328983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:29.329533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.329533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:29.329533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.329533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:29.345136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:29.345469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:29.345676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:29.345954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:29.346690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.346690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:29.347165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.347748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:29.347748 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.347748 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:29.347748 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:29.347748 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.347748 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:29.347748 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.347748 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:29.348766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.349116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:29.349650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.349650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:29.350071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:29.350924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:29.351365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:29.351365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:29.351365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:32.802806 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:32.804199 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:32.804367 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:32.809235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:32.900077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:32.900077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:32.900307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:32.900307 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:32.900307 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:32.900307 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:32.900307 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:32.900307 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:32.900307 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:32.900307 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.096 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:32.901483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:32.902005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:32.902339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:32.902339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:32.902976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:32.903418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:32.903931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:32.903931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:32.903931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:34.090925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:34.091387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:34.091387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:34.093139 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.093139 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-63423ddc-3191-453c-b834-f9c21753b809, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.094763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.094763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.094763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.094763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.094763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.095498 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.095498 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.095498 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.095498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.095686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:34.095686 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.095686 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:34.095686 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:34.095686 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.095686 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:34.095686 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.095686 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:34.096674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.097102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:34.097444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.097444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:34.097991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:34.099040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.100021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:34.100021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.100021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:34.105460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:34.106215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:34.106382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:34.106583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:34.107554 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.107936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.108766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:34.108766 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.108766 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:34.108766 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:34.108766 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.108766 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:34.108766 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.108766 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:34.109651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:34.110142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:34.110142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:34.110142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.110142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:34.110142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:34.110142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.110142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:34.110142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.110142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.110654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:34.111603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:34.111981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:34.112423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.112964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:34.323528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:34.323924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:34.324052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:34.324220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:34.324871 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.325190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.325923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:34.325923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.325923 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:34.325923 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:34.325923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.325923 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:34.325923 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.325923 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:34.326973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:34.327385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:34.327385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:34.327385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.327385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:34.327385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:34.327385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.327385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:34.327385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.327385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.328039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:34.328325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:34.328687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.328687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:34.328687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.328687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.329289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:34.329705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:34.330250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.330250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:34.330250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.330250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:34.348247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:34.348247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:34.348247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:34.348247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:34.349195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.349305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:34.349485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.350104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:34.350104 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.350104 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:34.350104 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:34.350104 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.350104 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:34.350104 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.350104 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:34.351895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.352321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:34.352692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.352692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:34.353597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:34.354032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:34.354695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:34.354695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:34.354695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:37.803675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:37.804614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:37.804614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:37.806067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:37.863917 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:37.864190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:37.864793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:37.864793 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:37.864793 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:37.864793 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:37.864793 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:37.864793 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:37.864793 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:37.864793 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.060 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:37.866685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:37.867095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:37.867481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:37.867993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:37.868670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:37.868670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:37.868670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:37.868670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:37.868670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:37.868670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:39.092769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:39.093316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:39.093316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:39.094789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.094972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.095180 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.095451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.095577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.095758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.096140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.096262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.096400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.096519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.100092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:39.100092 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.100092 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:39.100092 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:39.100092 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.100092 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:39.100092 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.100092 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:39.100990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.101462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:39.101833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.101833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:39.103710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:39.104214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.104779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:39.104779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.104779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:39.106344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:39.106660 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:39.106811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:39.106997 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:39.107532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.107778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.110666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:39.110666 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.110666 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:39.110666 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:39.110666 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.110666 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:39.110666 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.110666 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:39.112095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:39.112582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:39.112582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:39.112582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.112582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:39.112582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:39.112582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.112582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:39.112582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.112582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.113833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:39.114261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:39.114609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.114609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:39.114609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.114609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:39.117856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:39.118301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:39.118980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.118980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:39.118980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:39.118980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.118980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:39.118980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.118980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:39.325248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:39.327944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:39.327944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:39.328151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:39.334336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.334590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.335266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:39.335266 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.335266 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:39.335266 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:39.335266 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.335266 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:39.335266 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.335266 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:39.336058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:39.336528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:39.336528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:39.336528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.336528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:39.336528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:39.336528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.336528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:39.336528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.336528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.339502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:39.340397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:39.342023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:39.342581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.343233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:39.350555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:39.350555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:39.350555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:39.350555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:39.350555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.350555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:39.350858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.351994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:39.351994 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.351994 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:39.351994 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:39.351994 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.351994 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:39.351994 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.351994 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:39.353018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.353561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:39.353905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:39.354298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:39.354772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:39.354772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:39.354772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:39.354772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:39.354772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:39.354772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:42.804154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:42.804739 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:42.804739 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:42.805547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:42.892370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:42.892580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:42.893320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:42.893320 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:42.893320 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:42.893320 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:42.893320 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:42.893320 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:42.893320 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:42.893320 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.089 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:42.894400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:42.894695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:42.895007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:42.895007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:42.895489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:42.895919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:42.896265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:42.896265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:42.896265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:44.093316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:44.094944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:44.095146 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:44.098553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.099523 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.099749 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.099903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.100035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.100170 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.100293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.100436 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.100555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.100707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.105296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:44.105296 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.105296 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:44.105296 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:44.105296 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.105296 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:44.105296 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.105296 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:44.106201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.106797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:44.107086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.107086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:44.107431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:44.107871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.108913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:44.108913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.108913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:44.108913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:44.108913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:44.108913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:44.108913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:44.111163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.111443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.112427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:44.112427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.112427 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:44.112427 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:44.112427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.112427 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:44.112427 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.112427 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:44.113445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:44.113841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:44.113841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:44.113841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.113841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:44.113841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:44.113841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.113841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:44.113841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.113841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:44.114993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:44.115226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.115226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:44.115226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.115226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:44.117556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:44.117853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:44.118144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.118144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:44.118144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:44.118144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.118144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:44.118144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.118144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:44.327922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:44.327922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:44.328100 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:44.328348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:44.329021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.329238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.331509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:44.331509 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.331509 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:44.331509 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:44.331509 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.331509 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:44.331509 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.331509 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:44.332440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:44.332884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:44.332884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:44.332884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.332884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:44.332884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:44.332884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.332884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:44.332884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.332884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.334757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:44.335090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:44.335455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.335455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:44.335455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.335455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.336491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:44.337148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:44.337668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.337668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:44.337668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.337668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:44.349556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:44.350030 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:44.350119 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:44.350290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:44.353944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.353944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:44.353944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.354154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:44.354154 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.354154 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:44.354154 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:44.354154 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.354154 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:44.354154 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.354154 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:44.355047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.355363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:44.355715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.355715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:44.356779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:44.357185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:44.357817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:44.357817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:44.357817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:47.804655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:47.805158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:47.805158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:47.805872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:47.843734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:47.843947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:47.844420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:47.844420 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:47.844420 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:47.844420 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:47.844420 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:47.844420 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:47.844420 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:47.844420 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:47.845094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:47.845486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:47.845867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:47.846314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:47.846890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:47.846890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:47.846890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:47.846890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:47.846890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:47.846890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:49.095675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:49.095675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:49.095675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:49.096337 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.097719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.097719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.097719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.097719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.097719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.097978 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:49.098744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.099112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:49.099448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.099448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:49.100910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:49.101862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.102396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:49.102396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.102396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:49.108374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:49.108604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:49.108737 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:49.108857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:49.110688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.111059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:49.111532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:49.111892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:49.112246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.112612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.113710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:49.327962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:49.331933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:49.332156 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:49.332156 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:49.332424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.332603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.333152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:49.333152 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.333152 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:49.333152 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:49.333152 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.333152 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:49.333152 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.333152 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:49.333847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:49.334312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:49.334312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:49.334312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.334312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:49.334312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:49.334312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.334312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:49.334312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.334312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:49.335146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:49.335517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.335517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:49.335517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.335517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.335973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:49.336481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:49.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:49.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:49.350093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:49.350336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:49.350452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:49.350637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:49.350943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.351159 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:49.351440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.351934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:49.351934 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.351934 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:49.351934 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:49.351934 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.351934 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:49.351934 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.351934 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:49.352700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.352972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:49.353358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.353358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:49.353755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:49.354124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:49.354678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:49.354678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:49.354678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:52.805161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:52.805698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:52.805698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:52.806536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:52.819567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=242 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:05:52.859771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:52.860008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:52.860459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:52.860459 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:52.860459 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:52.860459 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:52.860459 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:52.860459 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:52.860459 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:52.860459 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:52.861249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:52.861611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:52.862116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:52.862116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:52.862384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:52.862774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:52.863102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:52.863102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:52.863102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:54.096376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:54.096376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:54.096376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:54.096837 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.097114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.097289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.097472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.097609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.097836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.097987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.098095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.098202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.098301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:54.100783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:54.101181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:54.101424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:54.101735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:54.102061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:54.102061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:54.102061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:54.102061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.102061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:54.102061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:54.102061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.102061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:54.102061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.102061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:54.109094 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:54.109724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:54.109724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:54.109724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:54.110203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.110389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.113169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:54.113521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:54.113862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.114287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:54.114561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:54.114841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:54.115152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:54.115152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:54.115152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.115152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:54.115152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:54.115152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.115152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:54.115152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.115152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:54.332906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:54.332906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:54.332906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:54.333649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:54.333649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.333889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.334516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:54.334516 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.334516 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:54.334516 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:54.334516 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.334516 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:54.334516 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.334516 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:54.335287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:54.335725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:54.335725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:54.335725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.335725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:54.335725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:54.335725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.335725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:54.335725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.335725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.336638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:54.337039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:54.337389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:54.337774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.338228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:54.350496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:54.355992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:54.356107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:54.356280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:54.357775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.357986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:54.358195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.358724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:54.358724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.358724 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:54.358724 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:54.358724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.358724 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:54.358724 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.358724 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:54.359755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.360149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:54.360412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.360412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:54.364714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:54.365237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:54.365769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:54.365769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:54.365769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:57.805703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:57.807975 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:57.808149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:57.809132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:57.849761 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:57.849984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:57.850578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:57.850578 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:57.850578 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:57.850578 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:57.850578 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:57.850578 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:57.850578 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:57.850578 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:57.851259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:57.851696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:57.852118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:57.852602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:57.853196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:57.853196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:57.853196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:57.853196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:57.853196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:57.853196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:59.095750 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:59.097748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:59.097748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:59.098196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.098484 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-31cd37db-91b9-408e-8710-2fcd8efd7c85, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.098703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.098864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.099012 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.099201 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.099341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.099476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.099610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a65a72b9-45ea-40eb-bbd1-b2a5a670b409, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.099767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:59.102890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:59.103393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:59.103780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:59.104160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:59.104759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:59.104759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:59.104759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:59.104759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.104759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:59.104759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:59.104759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.104759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:59.104759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.104759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:59.109618 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:59.110774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:59.111009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:59.111234 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:05:59.111835 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.112070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.115618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:59.116117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:59.116583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.116920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:59.117388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:59.117802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:59.118272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:59.118272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:59.118272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.118272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:59.118272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:59.118272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.118272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:59.118272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.118272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:59.332663 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:59.336013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:59.336223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:59.336223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:59.336888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.337350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.337987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:59.337987 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.337987 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:59.337987 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:59.337987 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.337987 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:59.337987 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.337987 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:05:59.338723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:05:59.339382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:59.339382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:05:59.339382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.339382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:05:59.339382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:05:59.339382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.339382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:05:59.339382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.339382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.339742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:59.340072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:59.340448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.340448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:59.340448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.340448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.342388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:59.346042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:59.346649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.346649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:59.346649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.346649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:05:59.356105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:05:59.356847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:05:59.356847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:05:59.357053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:05:59.357342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.358759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:05:59.358759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.359742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:05:59.359742 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.359742 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:05:59.359742 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:05:59.359742 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.359742 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:05:59.359742 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.359742 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:05:59.360920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.362177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:05:59.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:05:59.363137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:05:59.363697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:05:59.363697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:05:59.363697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:05:59.363697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:05:59.363697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:05:59.363697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:02.808044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:02.808535 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:02.808603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:02.809387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:02.874253 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:02.874614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:02.875351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:02.875351 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:02.875351 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:02.875351 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:02.875351 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:02.875351 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:02.875351 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:02.875351 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.067 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:02.876436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:02.877044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:02.877533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:02.877533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:02.877617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:02.878294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:02.878994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:02.878994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:02.878994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:04.099615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:04.099615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:04.099615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:04.101210 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.101533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.101730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.101961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.102129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.102313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.102460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.102601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.103301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:04.103301 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.103301 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:04.103301 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:04.103301 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.103301 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:04.103301 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.103301 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:04.104307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.105081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:04.105609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:04.106106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.106561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:04.106561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:04.106561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.106561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:04.106561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.106561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:04.111285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:04.111501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:04.111664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:04.111792 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:04.112273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.112708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.113211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:04.113211 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.113211 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:04.113211 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:04.113211 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.113211 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:04.113211 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.113211 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:04.113993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:04.114472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:04.114472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:04.114472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.114472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:04.114472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:04.114472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.114472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:04.114472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.114472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.114748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:04.115128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:04.115485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:04.115913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.116478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:04.336157 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:04.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:04.336850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:04.336941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:04.337933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.337963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.339221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:04.339221 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.339221 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:04.339221 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:04.339221 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.339221 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:04.339221 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.339221 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:04.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:04.340929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:04.340929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:04.340929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.340929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:04.340929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:04.340929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.340929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:04.340929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.340929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.341563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:04.341845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:04.342050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.342050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:04.342050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.342050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.344119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:04.345810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:04.346514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.346514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:04.346514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.346514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:04.356852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:04.357731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:04.358005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:04.358316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:04.358910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.359196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:04.359550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.360233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:04.360233 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.360233 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:04.360233 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:04.360233 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.360233 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:04.360233 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.360233 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:04.361053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.361316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:04.361657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.361657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:04.361830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:04.362156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:04.362684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:04.362684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:04.362684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:07.808774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:07.809540 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:07.809540 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:07.810365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:07.869269 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:07.870982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:07.870982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:07.870982 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:07.870982 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:07.870982 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:07.870982 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:07.870982 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:07.870982 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:07.870982 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:07.871597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:07.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:07.872120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:07.872120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:07.872163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:07.872524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:07.873009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:07.873009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:07.873009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:09.099482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:09.101050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:09.101261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:09.104142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.104414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.104687 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.104897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.105074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.105211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.105326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.105449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.105583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.106453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:09.106453 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.106453 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:09.106453 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:09.106453 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.106453 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:09.106453 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.106453 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:09.107414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.107887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:09.108263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.108263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:09.108451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:09.109008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.109530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:09.109530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.109530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:09.111666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:09.112063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:09.112247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:09.112364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:09.113131 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.113131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.113723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:09.113723 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.113723 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:09.113723 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:09.113723 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.113723 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:09.113723 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.113723 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:09.115350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:09.115677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:09.115677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:09.115677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.115677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:09.115677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:09.115677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.115677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:09.115677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.115677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.115949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:09.116197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:09.116436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.116436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:09.116436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.116436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:09.117159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:09.117524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:09.117852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.117852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:09.117852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:09.117852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.117852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:09.117852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.117852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:09.336830 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:09.339593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:09.339593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:09.339593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:09.339593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.339593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.339593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:09.339593 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.339593 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:09.339593 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:09.339972 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.339972 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:09.339972 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.339972 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:09.340311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:09.340782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:09.340782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:09.340782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.340782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:09.340782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:09.340782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.340782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:09.340782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.340782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.341361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:09.341599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:09.341919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.341919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:09.341919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.341919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.342345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:09.342727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:09.343278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.343278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:09.343278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.343278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:09.357705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:09.358087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:09.358314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:09.358498 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:09.359112 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.359456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:09.359886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.360476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:09.360476 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.360476 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:09.360476 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:09.360476 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.360476 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:09.360476 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.360476 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:09.361570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.362306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:09.362714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:09.363127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:09.363694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:09.363694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:09.363694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:09.363694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:09.363694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:09.363694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:12.810647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:12.810647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:12.810647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:12.811415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:12.867086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:12.867326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:12.868414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:12.868414 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:12.868414 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:12.868414 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:12.868414 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:12.868414 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:12.868414 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:12.868414 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.058 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:12.869361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:12.869797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:12.870221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:12.870221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:12.870882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:12.871352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:12.871896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:12.871896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:12.871896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:14.099946 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:14.100400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:14.100466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:14.101854 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.102579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.102579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.102713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.102902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.103078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.103243 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4903cac6-6e12-416c-ada2-ec5c4a648543, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.103422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.103577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.104322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:14.104322 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.104322 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:14.104322 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:14.104322 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.104322 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:14.104322 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.104322 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:14.105422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.105715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:14.106021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.106021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:14.106463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:14.106836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.107172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:14.107172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.107172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:14.112097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:14.112311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:14.112397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:14.112487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:14.113172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.113473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.114184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:14.114184 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.114184 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:14.114184 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:14.114184 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.114184 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:14.114184 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.114184 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:14.114913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:14.115197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:14.115197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:14.115197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.115197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:14.115197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:14.115197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.115197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:14.115197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.115197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.115553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:14.115829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:14.116056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.116056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:14.116056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.116056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:14.116289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:14.116691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:14.117004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.117004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:14.117004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:14.117004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.117004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:14.117004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.117004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:14.337664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:14.341823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:14.342046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:14.342264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:14.342763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.342995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.344017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:14.344017 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.344017 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:14.344017 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:14.344017 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.344017 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:14.344017 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.344017 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:14.353320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:14.353722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:14.353722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:14.353722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.353722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:14.353722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:14.353722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.353722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:14.353722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.353722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.364006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:14.364332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:14.364548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.364548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:14.364548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.364548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:14.365569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.367470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:14.368072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:14.368597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.368597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:14.368597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.368597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:14.369345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:14.371409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:14.371409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:14.371409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.374722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:14.374722 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.374722 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:14.374722 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:14.374722 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.374722 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:14.374722 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.374722 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:14.377096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.378933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:14.379278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:14.380886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:14.382516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:14.382516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:14.382516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:14.382516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:14.382516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:14.382516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:17.810930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:17.811529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:17.811595 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:17.817815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:17.866655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:17.866655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.056 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:17.868615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:17.868942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:17.869192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:17.869459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:17.869823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:17.869823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:17.869823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:17.869823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:17.869823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:17.869823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:17.869823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:17.869823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:17.869823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:19.100477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:19.101110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:19.101166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:19.102305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.102484 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.102599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.102742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.102859 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.102964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.103123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.103249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.103363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.103923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:19.103923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.103923 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:19.103923 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:19.103923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.103923 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:19.103923 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.103923 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:19.104711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.104954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:19.105175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.105175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:19.105482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:19.105905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.106339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:19.106339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.106339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:19.112965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:19.112965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:19.112965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:19.113035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:19.113559 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.114136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.114356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:19.114356 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.114356 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:19.114356 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:19.114356 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.114356 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:19.114356 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.114356 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:19.115946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:19.116238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:19.116238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:19.116238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.116238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:19.116238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:19.116238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.116238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:19.116238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.116238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.116702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:19.116913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:19.117143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.117143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:19.117143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.117143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:19.117420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:19.117711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:19.118035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.118035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:19.118035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:19.118035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.118035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:19.118035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.118035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:19.341928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:19.342355 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:19.342457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:19.342605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:19.343481 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.343481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.343645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:19.343645 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.343645 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:19.343645 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:19.343645 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.343645 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:19.343645 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.343645 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:19.344388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:19.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:19.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:19.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:19.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:19.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:19.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.345218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:19.345482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:19.345734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.345734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:19.345734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.345734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.346187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:19.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:19.346810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.346810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:19.346810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.346810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:19.368936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:19.369646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:19.369917 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:19.370297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:19.372206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.372206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:19.372206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.372543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:19.372543 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.372543 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:19.372543 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:19.372543 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.372543 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:19.372543 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.372543 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:19.374371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.374698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:19.374897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.374897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:19.375578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:19.375932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:19.376307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:19.376307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:19.376307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:22.811501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:22.811978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:22.811978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:22.812574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:22.855455 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:22.855687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:22.856456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:22.856456 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:22.856456 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:22.856456 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:22.856456 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:22.856456 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:22.856456 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:22.856456 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:22.857166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:22.857647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:22.858191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:22.858496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:22.858847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:22.858847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:22.858847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:22.858847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:22.858847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:22.858847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:24.100920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:24.101454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:24.101454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:24.102422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.102597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.103485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.103485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.103485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.103485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.103485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.103752 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.103752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.103935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:24.103935 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.103935 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:24.103935 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:24.103935 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.103935 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:24.103935 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.103935 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:24.104794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.105171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:24.106395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:24.106980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.107570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:24.107570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:24.107570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.107570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:24.107570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.107570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:24.112943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:24.113964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:24.113964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:24.113964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:24.114432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.114539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.117898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:24.118306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:24.118572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.118818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:24.119028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:24.119276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:24.119607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:24.119607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:24.119607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.119607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:24.119607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:24.119607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.119607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:24.119607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.119607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:24.342342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:24.343188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:24.343340 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:24.343483 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:24.343949 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.344147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.347099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:24.347099 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.347099 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:24.347099 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:24.347099 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.347099 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:24.347099 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.347099 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:24.348043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:24.348516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:24.348516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:24.348516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.348516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:24.348516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:24.348516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.348516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:24.348516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.348516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.349458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:24.349879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:24.350310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:24.350633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.350971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:24.369542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:24.369839 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:24.369965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:24.370141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:24.370525 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.372541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:24.373077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.374311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:24.374311 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.374311 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:24.374311 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:24.374311 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.374311 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:24.374311 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.374311 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:24.375096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.375494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:24.375838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.375838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:24.376129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:24.376582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:24.376930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:24.376930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:24.376930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:27.812184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:27.814973 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:27.815107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:27.815773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:27.868280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:27.868933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:27.870088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:27.870088 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:27.870088 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:27.870088 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:27.870088 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:27.870088 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:27.870088 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:27.870088 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:27.870939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:27.871499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:27.872052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:27.872591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:27.873120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:27.873120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:27.873120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:27.873120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:27.873120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:27.873120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:29.101807 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:29.102318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:29.102318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:29.103499 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.103705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.103853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.104000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.104129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.104253 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.104397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.104533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.104696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.104837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.105436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:29.105436 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.105436 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:29.105436 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:29.105436 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.105436 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:29.105436 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.105436 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:29.106596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.107196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:29.107484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.107484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:29.107692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:29.108077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.108540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:29.108540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.108540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:29.113668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:29.113956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:29.114066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:29.114206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:29.114691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.114910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.115416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:29.115416 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.115416 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:29.115416 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:29.115416 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.115416 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:29.115416 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.115416 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:29.116286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:29.116743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:29.116743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:29.116743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.116743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:29.116743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:29.116743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.116743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:29.116743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.116743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.117137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:29.117745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:29.118289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:29.118778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.119349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:29.343455 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:29.343972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:29.344105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:29.344296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:29.345690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.345690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.349539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.350016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.350967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:29.351253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:29.351560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:29.351878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.352248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:29.370014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:29.370387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:29.370520 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:29.370722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:29.371874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.371874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:29.371874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.372292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:29.372292 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.372292 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:29.372292 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:29.372292 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.372292 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:29.372292 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.372292 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:29.373162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.373458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:29.373725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.373725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:29.375431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:29.375766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:29.376155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:29.376155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:29.376155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:32.815244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:32.815692 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:32.815692 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:32.816954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:32.860529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:32.861494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:32.861529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:32.861529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:32.861529 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:32.861529 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:32.861529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:32.861529 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:32.861529 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:32.861529 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:32.862254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:32.862528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:32.862804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:32.862804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:32.863014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:32.863450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:32.863823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:32.863823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:32.863823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:34.102220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:34.102726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:34.102778 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:34.105467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.105467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.105467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.105467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.105467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.105935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.105935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.105935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.105935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.105935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.105935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:34.105935 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.105935 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:34.105935 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.106218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:34.106615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:34.106931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.106931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:34.106931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.106931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:34.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:34.107312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.107700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:34.107700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.107700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:34.114034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:34.114265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:34.114350 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:34.114448 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:34.114794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.115047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.115644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:34.115644 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.115644 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:34.115644 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:34.115644 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.115644 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:34.115644 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.115644 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:34.116207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:34.116549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:34.116549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:34.116549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.116549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:34.116549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:34.116549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.116549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:34.116549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.116549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.116839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:34.117058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:34.117319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.117319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:34.117319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.117319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:34.117531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:34.117880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:34.118351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.118351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:34.118351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:34.118351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.118351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:34.118351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.118351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:34.344063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:34.344464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:34.344609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:34.344794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:34.345254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.345495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.346073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:34.346073 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.346073 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:34.346073 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:34.346073 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.346073 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:34.346073 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.346073 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:34.347042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:34.347462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:34.347462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:34.347462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.347462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:34.347462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:34.347462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.347462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:34.347462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.347462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.348122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:34.348493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:34.348916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.348916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:34.348916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.348916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.349248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:34.349962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:34.350842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.350842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:34.350842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.350842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:34.370517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:34.370813 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:34.370969 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:34.371101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:34.371460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.372100 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:34.372486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.373089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:34.373089 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.373089 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:34.373089 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:34.373089 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.373089 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:34.373089 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.373089 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:34.373956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.374386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:34.374786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.374786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:34.375224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:34.375868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:34.376408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:34.376408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:34.376408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:37.815848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:37.816287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:37.816287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:37.817015 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:37.862757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:37.862757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:37.862757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:37.862757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:37.862757 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:37.862757 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:37.862757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:37.862757 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:37.862757 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:37.862757 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:37.863330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:37.863595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:37.863871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:37.863871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:37.864069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:37.864404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:37.864794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:37.864794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:37.864794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:39.105658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:39.105658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:39.105658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:39.105658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.105658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.106208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.106208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.106208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.106208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.106208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:39.106315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:39.106522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.106794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:39.107028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:39.107274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.107554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:39.107554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.107554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:39.117466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:39.117466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:39.117466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:39.117466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:39.117466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.117466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.117965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:39.118751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:39.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.119843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:39.120109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:39.120362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:39.120663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:39.120663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:39.120663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.120663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:39.120663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:39.120663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.120663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:39.120663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.120663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:39.344637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:39.345304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:39.345304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:39.345384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:39.346325 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.346325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.346950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:39.346950 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.346950 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:39.346950 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:39.346950 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.346950 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:39.346950 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.346950 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:39.350650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:39.351236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:39.351576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.351935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.356970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:39.357963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:39.358525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.358525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:39.358525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.358525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:39.371575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:39.376466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:39.376542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:39.377043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:39.377429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.378641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:39.378641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.383125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:39.383125 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.383125 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:39.383125 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:39.383125 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.383125 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:39.383125 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.383125 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:39.384149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.384558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:39.384974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.384974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:39.385061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:39.385503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:39.386089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:39.386089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:39.386089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:42.817958 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:42.818450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:42.818450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:42.819204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:42.871905 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:42.872091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:42.874651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:42.875356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:42.876031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:42.876356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:42.876794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:42.876794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:42.876794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:42.876794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:42.876794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:42.876794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:42.876794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:42.876794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:42.876794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:44.103174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:44.103672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:44.103672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:44.104876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.105074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.105238 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.105375 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.105496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.105639 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.105773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.105895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.106032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.106158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.106744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:44.106744 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.106744 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:44.106744 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:44.106744 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.106744 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:44.106744 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.106744 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:44.107772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.108233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:44.108579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.108579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:44.108937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:44.109499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.110057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:44.110057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.110057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:44.115289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:44.115541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:44.115711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:44.115853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:44.116547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.116753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.117273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:44.117273 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.117273 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:44.117273 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:44.117273 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.117273 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:44.117273 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.117273 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:44.117815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:44.118267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:44.118267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:44.118267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.118267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:44.118267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:44.118267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.118267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:44.118267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.118267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.118515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:44.118927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:44.119327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:44.119737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.120244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:44.347125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:44.347125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:44.347125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:44.347125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:44.347125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.347877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.347877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:44.347877 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.347877 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:44.347877 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:44.347877 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.347877 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:44.347877 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.347877 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:44.348730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:44.349834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:44.351093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:44.351507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:44.352067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.352591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:44.376638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:44.377083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:44.377212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:44.377384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:44.377730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.377950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:44.378239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.378915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:44.378915 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.378915 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:44.378915 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:44.378915 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.378915 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:44.378915 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.378915 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:44.379565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.379824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:44.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:44.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:44.381804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:44.381804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:44.381804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:44.381804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:44.381804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:44.381804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:47.818503 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:47.819565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:47.819565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:47.821214 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:47.851608 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:47.851840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:47.853751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:47.854055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:47.854303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:47.854570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:47.854909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:47.854909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:47.854909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:47.854909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:47.854909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:47.854909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:47.854909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:47.854909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:47.854909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:49.104041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:49.107464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:49.107464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:49.111101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.111101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.111101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.111101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.111101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.111312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.113942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.113942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.113942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.113942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.115494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:49.115494 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.115494 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:49.115494 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:49.115494 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.115494 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:49.115494 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.115494 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:49.117220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.117691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:49.118091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.118091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:49.119731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:49.120166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.120653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:49.120653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.120653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:49.120653 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:49.121423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:49.121423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:49.121423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:49.124506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.124723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.127179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:49.127542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:49.127853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.128051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:49.128291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:49.128687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:49.128964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:49.128964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:49.128964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.128964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:49.128964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:49.128964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.128964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:49.128964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.128964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:49.348951 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:49.349439 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:49.349604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:49.349806 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:49.350205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.350480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.353299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:49.353299 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.353299 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:49.353299 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:49.353299 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.353299 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:49.353299 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.353299 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:49.354216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:49.354777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:49.354777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:49.354777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.354777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:49.354777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:49.354777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.354777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:49.354777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.354777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.355204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:49.355549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:49.356385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:49.356785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.357376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:49.377153 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:49.377703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:49.377920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:49.378172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:49.378536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.378815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:49.379136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.379552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:49.379552 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.379552 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:49.379552 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:49.379552 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.379552 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:49.379552 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.379552 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:49.380295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.380742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:49.381110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.381110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:49.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:49.381688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:49.382773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:49.382773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:49.382773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:52.819357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:52.819821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:52.819821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:52.820440 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:52.827636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=238 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:06:52.869641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:52.869929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:52.872575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:52.873496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:52.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:52.874548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:52.875185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:52.875185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:52.875185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:52.875185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:52.875185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:52.875185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:52.875185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:52.875185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:52.875185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:54.106243 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:54.108047 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:54.108047 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:54.108097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.108439 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.108603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.108794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.108950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.109107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.109265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.109422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.109587 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.109774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:54.112390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:54.112794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:54.113138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:54.113469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:54.114091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:54.114091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:54.114091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:54.114091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.114091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:54.114091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:54.114091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.114091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:54.114091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.114091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:54.122007 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:54.122467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:54.122746 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:54.122957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:54.123504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.125161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.125767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:54.125767 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.125767 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:54.125767 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:54.125767 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.125767 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:54.125767 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.125767 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:54.127399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:54.127914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:54.128315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.128681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:54.129039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:54.129382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:54.129671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.129671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:54.129671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:54.129671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.129671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:54.129671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.129671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:54.349428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:54.350653 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:54.351920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:54.351920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:54.352106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.352324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.356891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:54.356891 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.356891 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:54.356891 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:54.356891 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.356891 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:54.356891 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.356891 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:54.359806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:54.360494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:54.360833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.361104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.362264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:54.362676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:54.363075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.363075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:54.363075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.363075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:54.377981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:54.378128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:54.378273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:54.378441 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:54.378906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.379167 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:54.379546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.380202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:54.380202 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.380202 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:54.380202 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:54.380202 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.380202 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:54.380202 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.380202 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:54.381455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:54.383135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:54.383749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:54.384578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:54.384578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:54.384578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:54.384578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:54.384578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:54.384578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:57.820855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:57.820855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:57.820855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:57.821609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:57.917133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:57.917133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:57.917585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:57.917585 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:57.917585 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:57.917585 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:57.917585 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:57.917585 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:57.917585 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:57.917585 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.097 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:57.919172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:57.919503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:57.919850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:57.919850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:57.920943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:57.921375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:57.921881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:57.921881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:57.921881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:59.107150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:59.108063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:59.109052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:59.110099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.110439 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.110669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.111077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.111130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.111297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.111442 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.111579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.111744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.111930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:59.114180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:59.114759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:59.115139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:59.115544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:59.116102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:59.116102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:59.116102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:59.116102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.116102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:59.116102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:59.116102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.116102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:59.116102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.116102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:59.122594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:59.123312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:59.123736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:59.123935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:06:59.124634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.124910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.132806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:59.132806 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.132806 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:59.132806 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:59.132806 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.132806 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:59.132806 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.132806 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:59.134375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:59.135616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:59.135616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:59.135616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.135616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:59.135616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:59.135616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.135616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:59.135616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.135616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.138007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:59.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:59.140060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:59.140528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.140836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:59.350570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:59.350999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:59.351145 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:59.351272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:59.353963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.354161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.354688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:59.354688 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.354688 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:59.354688 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:59.354688 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.354688 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:59.354688 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.354688 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:06:59.355401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:06:59.355777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:59.355777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:06:59.355777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.355777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:06:59.355777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:06:59.355777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.355777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:06:59.355777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.355777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.356427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:59.356652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:59.356921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.356921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:59.356921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.356921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.357470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:59.357828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:59.358148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.358148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:59.358148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.358148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:06:59.378154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:06:59.381049 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:06:59.381157 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:06:59.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:06:59.381633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.381965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:06:59.382544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.382679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:06:59.382679 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.382679 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:06:59.382679 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:06:59.382679 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.382679 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:06:59.382679 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.382679 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:06:59.383336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.383564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:06:59.383779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.383779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:06:59.384186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:06:59.384542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:06:59.384854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:06:59.384854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:06:59.384854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:02.820233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:02.820893 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:02.821170 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:02.822038 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:02.873109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:02.873371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:02.874164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:02.874164 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:02.874164 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:02.874164 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:02.874164 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:02.874164 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:02.874164 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:02.874164 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:02.875259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:02.875592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:02.875923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:02.875923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:02.876428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:02.876871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:02.877378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:02.877378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:02.877378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:04.108276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:04.109575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:04.110776 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:04.114029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.114029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-5248ba76-4295-4afb-ba06-c52d73d8034a, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.114029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.114029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.114029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.114524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.114524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.114524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.114524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.114524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.114935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:04.114935 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.114935 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:04.114935 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:04.114935 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.114935 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:04.114935 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.114935 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:04.116276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.116750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:04.117236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.117236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:04.118259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:04.119713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.120385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:04.120385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.120385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:04.123372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:04.123372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:04.123372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.126088 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:04.127820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:04.128913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:04.129276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.131612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:04.132350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:04.132975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:04.133336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.133336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:04.133336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:04.133336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.133336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:04.133336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.133336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:04.351563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:04.352868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:04.353010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:04.353173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:04.354682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.354682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.355075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:04.355075 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.355075 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:04.355075 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:04.355075 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.355075 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:04.355075 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.355075 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:04.356211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:04.356771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:04.356771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:04.356771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.356771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:04.356771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:04.356771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.356771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:04.356771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.356771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.357758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:04.357981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:04.358250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.358250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:04.358250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.358250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.363951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:04.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:04.365731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.365731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:04.365731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.365731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:04.381464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:04.381905 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:04.382045 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:04.382235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:04.382857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.383130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:04.383478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.384600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:04.384600 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.384600 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:04.384600 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:04.384600 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.384600 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:04.384600 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.384600 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:04.386570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.386999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:04.387334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.387334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:04.387756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:04.388318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:04.388664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:04.388664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:04.388664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:07.820919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:07.823722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:07.823722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:07.823722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:07.903931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:07.904263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.084 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:07.908277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:07.909242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:07.909763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:07.910345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:07.911008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:07.911008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:07.911008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:07.911008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:07.911008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:07.911008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:07.911008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:07.911008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:07.911008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:09.108856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:09.109267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:09.109307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:09.110778 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.110778 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.110894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.110986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.111140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.111297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.111426 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.111551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.111724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.112337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:09.112337 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.112337 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:09.112337 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:09.112337 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.112337 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:09.112337 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.112337 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:09.113073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.113391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:09.113609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.113609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:09.113830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:09.114117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.114416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:09.114416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.114416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:09.125227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:09.125650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:09.126082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:09.126316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:09.126834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.127036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.127649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:09.127649 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.127649 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:09.127649 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:09.127649 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.127649 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:09.127649 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.127649 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:09.128201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:09.128599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:09.128599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:09.128599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.128599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:09.128599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:09.128599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.128599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:09.128599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.128599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.128907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:09.129227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:09.129616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:09.130092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.130711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:09.353026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:09.356360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:09.356360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:09.356506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:09.356900 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.357119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.360882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:09.360882 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.360882 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:09.360882 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:09.360882 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.360882 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:09.360882 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.360882 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:09.361743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:09.362079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:09.362079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:09.362079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.362079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:09.362079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:09.362079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.362079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:09.362079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.362079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.362712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:09.363077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:09.363425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.363425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:09.363425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.363425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.371707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:09.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:09.372710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.372710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:09.372710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.372710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:09.381883 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:09.383070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:09.383070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:09.383070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:09.383070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.383313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:09.383545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.384192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:09.384192 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.384192 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:09.384192 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:09.384192 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.384192 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:09.384192 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.384192 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:09.385042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.385431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:09.385819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:09.386232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:09.386893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:09.386893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:09.386893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:09.386893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:09.386893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:09.386893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:12.822110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:12.822615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:12.824649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:12.824649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:12.882570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:12.882857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:12.883489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:12.883489 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:12.883489 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:12.883489 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:12.883489 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:12.883489 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:12.883489 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:12.883489 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:12.884499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:12.884806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:12.885044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:12.885044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:12.885702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:12.886133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:12.886457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:12.886457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:12.886457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:14.109385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:14.109385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:14.109385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:14.115894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.116182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.116953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.116953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.116953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.116953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.116953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.117245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.117245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.117245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:14.121711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:14.122994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.123326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:14.123326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:14.123326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.123326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:14.123326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.123326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:14.124578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:14.125063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.125512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:14.125512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.125512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:14.126099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:14.126880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:14.126880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:14.126880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:14.132278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.133560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.134009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:14.134009 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.134009 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:14.134009 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:14.134009 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.134009 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:14.134009 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.134009 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:14.135995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:14.136488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:14.136950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.137642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:14.138005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:14.138469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:14.139318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.139318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:14.139318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:14.139318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.139318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:14.139318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.139318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:14.359018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:14.359742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:14.359742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:14.359742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:14.360190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.360464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.364205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:14.364829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:14.365233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.365585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:14.365857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:14.366099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:14.368032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:14.368032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.368032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:14.368032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:14.368032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.368032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:14.368032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.368032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:14.382870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:14.382870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:14.382870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:14.384254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:14.385116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.385519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:14.385864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.386456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:14.386456 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.386456 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:14.386456 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:14.386456 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.386456 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:14.386456 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.386456 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:14.387369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.387657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:14.387882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.387882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:14.390311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:14.390584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:14.390983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:14.390983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:14.390983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:17.823178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:17.823649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:17.823649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:17.824647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:17.890617 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:17.890898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:17.892054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:17.892054 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:17.892054 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:17.892054 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:17.892054 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:17.892054 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:17.892054 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:17.892054 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.068 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:17.893968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:17.894383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:17.894776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:17.895225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:17.895751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:17.895751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:17.895751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:17.895751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:17.895751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:17.895751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:19.109923 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:19.111774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:19.111774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:19.112128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.112302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.112397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.112531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.112763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.112842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.113002 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.113132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.113224 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.113322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:19.116604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:19.117281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:19.117733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:19.118258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:19.118901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:19.118901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:19.118901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:19.118901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.118901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:19.118901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:19.118901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.118901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:19.118901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.118901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:19.126641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:19.128845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:19.129336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:19.130233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:19.133127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.133348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.134001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:19.134001 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.134001 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:19.134001 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:19.134001 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.134001 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:19.134001 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.134001 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:19.136584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:19.137480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:19.137480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:19.137480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.137480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:19.137480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:19.137480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.137480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:19.137480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.137480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.138798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:19.139235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:19.139606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.139606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:19.139606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.139606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:19.141800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:19.142326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:19.142774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.142774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:19.142774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:19.142774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.142774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:19.142774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.142774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:19.360232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:19.361142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:19.361142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:19.361142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:19.363310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.363574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.367657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:19.368210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:19.368604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.368942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:19.369304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:19.369730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:19.370238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:19.370238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.370238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:19.370238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:19.370238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.370238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:19.370238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.370238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:19.385383 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:19.386362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:19.386362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:19.386362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:19.387384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.387691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:19.388017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.390216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:19.390216 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.390216 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:19.390216 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:19.390216 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.390216 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:19.390216 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.390216 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:19.393715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.394091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:19.394673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:19.395201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:19.395802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:19.395802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:19.395802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:19.395802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:19.395802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:19.395802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:22.823339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:22.823780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:22.823933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:22.824753 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:22.888584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:22.889179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:22.889957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:22.889957 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:22.889957 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:22.889957 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:22.889957 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:22.889957 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:22.889957 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:22.889957 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.066 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:22.893687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:22.894436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:22.894930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:22.895439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:22.896464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:22.896464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:22.896464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:22.896464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:22.896464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:22.896464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:24.110987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:24.111395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:24.111395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:24.112507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.112742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.112896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.113044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.113195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.113334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.113475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.113612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e1f46f07-f658-44a9-af7f-254eb7a2d8b8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.113762 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.113903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:24.114494 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.114494 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:24.114494 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:24.114494 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.114494 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:24.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.114494 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:24.115407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.115740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:24.116095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.116095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:24.116556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:24.116846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.117217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:24.117217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.117217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:24.127341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:24.127609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:24.127745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:24.127870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:24.128242 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.128424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.128982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:24.128982 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.128982 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:24.128982 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:24.128982 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.128982 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:24.128982 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.128982 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:24.129710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:24.130047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:24.130047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:24.130047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.130047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:24.130047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:24.130047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.130047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:24.130047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.130047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.130413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:24.130779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:24.131250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.131250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:24.131250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.131250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:24.131364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:24.131788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:24.132250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.132250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:24.132250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:24.132250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.132250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:24.132250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.132250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:24.361025 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:24.361309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:24.361392 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:24.361506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:24.361901 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.362148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.362725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:24.362725 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.362725 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:24.362725 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:24.362725 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.362725 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:24.362725 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.362725 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:24.363327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:24.363671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:24.363671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:24.363671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.363671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:24.363671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:24.363671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.363671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:24.363671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.363671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.363925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:24.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:24.364381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.364381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:24.364381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.364381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.364724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:24.365211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:24.365914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.365914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:24.365914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.365914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:24.385953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:24.386361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:24.386453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:24.386595 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:24.386952 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.387174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:24.387514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.387906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:24.387906 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.387906 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:24.387906 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:24.387906 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.387906 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:24.387906 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.387906 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:24.388511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.388914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:24.389263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:24.389707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:24.390255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:24.390255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:24.390255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:24.390255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:24.390255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:24.390255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:27.823962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:27.824860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:27.824860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:27.825736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:27.888346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:27.888808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:27.889465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:27.889465 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:27.889465 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:27.889465 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:27.889465 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:27.889465 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:27.889465 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:27.889465 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.065 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:27.892203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:27.893809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:27.894506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:27.895399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:27.896522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:27.896522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:27.896522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:27.896522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:27.896522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:27.896522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:29.112664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:29.113146 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:29.113146 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:29.115087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.115261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.115374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.115478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.115581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.115696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.115796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.117265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.117384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.117895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:29.117895 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.117895 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:29.117895 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:29.117895 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.117895 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:29.117895 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.117895 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:29.121391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.121911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:29.122264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:29.122656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.123084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:29.123084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:29.123084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.123084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:29.123084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.123084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:29.127763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:29.128177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:29.128319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:29.128451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:29.128803 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.129334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.129442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:29.129442 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.129442 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:29.129442 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:29.129442 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.129442 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:29.129442 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.129442 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:29.130263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:29.130705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:29.130705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:29.130705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.130705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:29.130705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:29.130705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.130705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:29.130705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.130705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.131646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:29.132044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:29.132378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:29.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.135533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:29.361478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:29.363432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:29.365108 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:29.368133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:29.369124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.369422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.371719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:29.371719 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.371719 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:29.371719 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:29.371719 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.371719 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:29.371719 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.371719 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:29.372521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:29.372941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:29.372941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:29.372941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.372941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:29.372941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:29.372941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.372941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:29.372941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.372941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.373720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:29.374163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:29.374509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.374509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:29.374509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.374509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.376122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:29.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:29.377026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.377026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:29.377026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.377026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:29.386443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:29.387382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:29.387507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:29.387684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:29.389186 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.389410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:29.389609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.393318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:29.393318 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.393318 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:29.393318 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:29.393318 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.393318 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:29.393318 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.393318 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:29.394727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.395126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:29.395458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.395458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:29.395596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:29.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:29.396700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:29.396700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:29.396700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:32.824529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:32.825004 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:32.825004 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:32.825980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:32.887154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:32.887368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:32.889990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:32.890326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:32.890570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:32.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:32.891237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:32.891237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:32.891237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:32.891237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:32.891237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:32.891237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:32.891237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:32.891237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:32.891237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:34.119407 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:34.121931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:34.121931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:34.122518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.122518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.122518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.122518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.122518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.122989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.122989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.122989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.122989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.122989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.123596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:34.123596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.123596 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:34.123596 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:34.123596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.123596 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:34.123596 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.123596 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:34.125707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.126057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:34.126401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.126401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:34.128260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:34.129477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.130939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:34.130939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.130939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:34.130939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:34.130939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:34.133892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:34.133892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:34.134931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.135091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.138642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:34.139152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:34.139578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.139942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:34.140294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:34.140759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:34.141239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:34.141239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:34.141239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.141239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:34.141239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:34.141239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.141239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:34.141239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.141239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:34.362078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:34.363237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:34.363237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:34.363237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:34.363727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.364615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.367977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:34.368586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:34.369011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.369395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:34.369794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.369794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.372505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:34.374077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:34.375488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.375488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:34.375488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.375488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:34.389039 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:34.390424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:34.390424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:34.390424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:34.393800 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.394126 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:34.394868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.396885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:34.396885 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.396885 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:34.396885 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:34.396885 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.396885 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:34.396885 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.396885 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:34.401611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.402167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:34.402653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:34.403054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:34.403549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:34.403549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:34.403549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:34.403549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:34.403549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:34.403549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:37.825086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:37.825556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:37.825919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:37.826655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:37.887806 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:37.889344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:37.889344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:37.889344 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:37.889344 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:37.889344 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:37.889344 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:37.889344 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:37.889344 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:37.889344 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:37.890403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:37.890837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:37.891248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:37.891776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:37.892372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:37.892372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:37.892372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:37.892372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:37.892372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:37.892372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:39.120834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:39.121385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:39.121385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:39.122659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.122920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.123101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.123276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.123430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.123684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.123821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.123950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.124076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.124201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.125011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:39.125011 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.125011 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:39.125011 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:39.125011 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.125011 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:39.125011 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.125011 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:39.126345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.126886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:39.127275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.127275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:39.128611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:39.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.129878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:39.129878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.129878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:39.130876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:39.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:39.131946 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:39.132335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:39.132891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.133180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.134061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:39.134061 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.134061 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:39.134061 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:39.134061 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.134061 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:39.134061 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.134061 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:39.135102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:39.135755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:39.135755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:39.135755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.135755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:39.135755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:39.135755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.135755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:39.135755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.135755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.136855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:39.137201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:39.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:39.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:39.139556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:39.140051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:39.140550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.140550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:39.140550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:39.140550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.140550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:39.140550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.140550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:39.362701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:39.362919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:39.363018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:39.363143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:39.363639 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.363921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.364455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:39.364455 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.364455 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:39.364455 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:39.364455 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.364455 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:39.364455 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.364455 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:39.365130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:39.365370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:39.365370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:39.365370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.365370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:39.365370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:39.365370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.365370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:39.365370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.365370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.365829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:39.366106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:39.366322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.366322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:39.366322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.366322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.366668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:39.367238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:39.367901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.367901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:39.367901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.367901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:39.389516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:39.390824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:39.391056 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:39.391218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:39.391935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.392139 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:39.392488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.393122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:39.393122 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.393122 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:39.393122 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:39.393122 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.393122 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:39.393122 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.393122 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:39.394491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.394945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:39.395296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:39.395768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:39.396249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:39.396249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:39.396249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:39.396249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:39.396249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:39.396249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:42.826033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:42.826033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:42.826033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:42.826613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:42.899950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:42.900160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:42.900785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:42.900785 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:42.900785 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:42.900785 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:42.900785 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:42.900785 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:42.900785 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:42.900785 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.075 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:42.901677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:42.901954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:42.902207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:42.902207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:42.902869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:42.903177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:42.903505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:42.903505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:42.903505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:44.121339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:44.121789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:44.121789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:44.123225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.123452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.123656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.123831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.123987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.124133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.124276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.124420 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.124610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.124755 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.124916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.125465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:44.125465 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.125465 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:44.125465 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:44.125465 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.125465 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:44.125465 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.125465 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:44.126407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.126955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:44.127312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.127312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:44.127431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:44.127947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.128549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:44.128549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.128549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:44.131778 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:44.132243 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:44.132430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:44.132637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:44.134419 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.134419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.134419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:44.134419 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.134419 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:44.134419 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:44.134419 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.134419 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:44.134419 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.134419 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:44.135292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:44.135826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:44.135826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:44.135826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.135826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:44.135826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:44.135826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.135826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:44.135826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.135826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.137091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:44.137528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:44.137947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.137947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:44.137947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.137947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:44.138350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:44.138817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:44.139346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.139346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:44.139346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:44.139346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.139346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:44.139346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.139346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:44.363191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:44.363444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:44.363739 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:44.364008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:44.364799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.364971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.365457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:44.365457 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.365457 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:44.365457 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:44.365457 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.365457 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:44.365457 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.365457 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:44.366230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:44.366817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:44.366817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:44.366817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.366817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:44.366817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:44.366817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.366817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:44.366817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.366817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.367113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:44.367516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:44.367924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:44.368325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.368978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:44.391040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:44.392008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:44.392008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:44.392130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:44.392617 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.392902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:44.393228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.394194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:44.394194 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.394194 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:44.394194 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:44.394194 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.394194 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:44.394194 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.394194 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:44.395317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.395832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:44.396258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.396258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:44.397140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:44.397765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:44.398516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:44.398516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:44.398516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:47.826327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:47.826327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:47.826970 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:47.827320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:47.865306 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:47.865534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:47.866168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:47.866168 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:47.866168 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:47.866168 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:47.866168 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:47.866168 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:47.866168 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:47.866168 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:47.867124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:47.867546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:47.867984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:47.868452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:47.869000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:47.869000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:47.869000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:47.869000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:47.869000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:47.869000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:49.121879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:49.122479 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:49.122479 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:49.123948 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.124125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.124279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.124427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.124574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.124742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.124895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.125030 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.125165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.125310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4b97864f-51cf-416c-8a3b-f3989bc16eb8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.125429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.126263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:49.126263 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.126263 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:49.126263 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:49.126263 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.126263 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:49.126263 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.126263 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:49.127508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.127899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:49.128281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.128281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:49.128790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:49.129238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.129816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:49.129816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.129816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:49.132368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:49.132765 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:49.132918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:49.133057 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:49.133825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.135106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.136146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:49.136146 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.136146 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:49.136146 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:49.136146 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.136146 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:49.136146 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.136146 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:49.136680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:49.136989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:49.136989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:49.136989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.136989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:49.136989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:49.136989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.136989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:49.136989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.136989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.137329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:49.137587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:49.137896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.137896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:49.137896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.137896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:49.138218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:49.138537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:49.138911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.138911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:49.138911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:49.138911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.138911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:49.138911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.138911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:49.363742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:49.364518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:49.364731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:49.364918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:49.366033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.366244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.369555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:49.369555 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.369555 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:49.369555 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:49.369555 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.369555 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:49.369555 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.369555 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:49.370406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:49.370928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:49.370928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:49.370928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.370928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:49.370928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:49.370928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.370928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:49.370928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.370928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.371165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:49.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:49.371944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.371944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:49.371944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.371944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.372178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:49.372733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:49.373328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.373328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:49.373328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.373328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:49.392650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:49.392650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:49.393002 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:49.393766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:49.393766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.394849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:49.394849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.395744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:49.395744 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.395744 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:49.395744 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:49.395744 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.395744 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:49.395744 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.395744 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:49.396752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.397043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:49.397347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.397347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:49.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:49.398806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:49.399228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:49.399228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:49.399228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:52.826661 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:52.828933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:52.829810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:52.834151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:52.837554 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=239 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:07:52.889368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:52.889638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:52.890368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:52.890368 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:52.890368 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:52.890368 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:52.890368 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:52.890368 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:52.890368 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:52.890368 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:52.891450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:52.891917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:52.892235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:52.892235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:52.892573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:52.893133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:52.893593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:52.893593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:52.893593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:54.124973 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:54.125742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:54.125947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:54.127698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.127966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.128117 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.128255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.128383 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.130326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.130487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.130605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.130744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.130887 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-89b49445-3ec8-4b4b-93b2-3786dad022ab, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.130997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.131650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:54.131650 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.131650 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:54.131650 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:54.131650 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.131650 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:54.131650 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.131650 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:54.132669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.133011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:54.133293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.133293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:54.134179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:54.134825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.135307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:54.135307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.135307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:54.135307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:54.135307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:54.135693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:54.137432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:54.137432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.137432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.137920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:54.137920 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.137920 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:54.137920 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:54.137920 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.137920 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:54.137920 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.137920 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:54.138946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:54.139349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:54.139349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:54.139349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.139349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:54.139349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:54.139349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.139349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:54.139349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.139349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.140703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:54.141143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:54.141437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:54.141744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.142103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:54.364742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:54.365492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:54.365492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:54.365492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:54.367556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.367556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.367950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:54.367950 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.367950 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:54.367950 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:54.367950 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.367950 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:54.367950 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.367950 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:54.369156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:54.369609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:54.369609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:54.369609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.369609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:54.369609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:54.369609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.369609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:54.369609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.369609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.371845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:54.372215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:54.372530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.372530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:54.372530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.372530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.374531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:54.375100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:54.375539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.375539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:54.375539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.375539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:54.392255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:54.397023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:54.398980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:54.398980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:54.398980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.398980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:54.398980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.400426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:54.400426 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.400426 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:54.400426 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:54.400426 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.400426 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:54.400426 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.400426 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:54.404692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.405299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:54.405604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.405604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:54.406083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:54.406523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:54.408072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:54.408072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:54.408072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:57.815172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:57.815605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:57.815605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:57.828931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:57.830712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 27 16:07:57.830897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Number of inconsistencies found at create/update: networks=7, security_groups=3, routers=4, router_ports=4 {{(pid=92285) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 27 16:07:57.831044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Number of inconsistencies found at delete: router_ports=16 {{(pid=92285) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 27 16:07:57.831174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1a1659c0-4943-4985-b3f1-75f0c26af59f (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:57.878095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, resource={'id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'name': 'public', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['156facd3-ecc0-4e78-8b24-b1c2e5631a78', '3c538124-e9fa-4f89-905c-f13de628c91d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-27T15:57:46Z', 'updated_at': '2026-05-27T15:57:53Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.878353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.878673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1d74a028-7065-4269-8c77-c99e0ea79147, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.878868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.879336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:57.879336 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:57.879336 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:57.879336 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:57.879336 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:57.879336 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:57.879336 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.879336 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 1a1659c0-4943-4985-b3f1-75f0c26af59f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:57.880107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:57.880497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:57.880936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:57.880936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.880936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:57.880936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 28588915-92a2-4ece-bd6f-c59cf1c236ce (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:57.907034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28588915-92a2-4ece-bd6f-c59cf1c236ce, resource={'id': '28588915-92a2-4ece-bd6f-c59cf1c236ce', 'name': 'shared', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['73ef14d1-56af-457d-a4cc-cd982d0f7719'], 'standard_attr_id': 26, 'shared': True, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T15:58:53Z', 'updated_at': '2026-05-27T15:58:54Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 65356}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.907206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28588915-92a2-4ece-bd6f-c59cf1c236ce, col_values=(('external_ids', {'neutron:network_name': 'shared', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.907447 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-854ee673-4c8a-4fe6-bb8f-35261288982d, external_ids_update=None, columns={'tag': 65356}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.907865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.907966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:57.907966 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:57.907966 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:57.907966 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:57.907966 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:57.907966 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:57.907966 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.907966 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 28588915-92a2-4ece-bd6f-c59cf1c236ce (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:57.908797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:57.911416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:57.911861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:57.911861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.911861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:57.911861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 26791013-51e5-4bfa-9444-c469ccd1f068 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:57.933219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-26791013-51e5-4bfa-9444-c469ccd1f068, resource={'id': '26791013-51e5-4bfa-9444-c469ccd1f068', 'name': 'tempest-ServerActionsTestOtherA-1037644522-network', 'tenant_id': '8245a9a47a0544aa865dde98974ae162', 'project_id': '8245a9a47a0544aa865dde98974ae162', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['859e1e3c-d016-4c6f-bc69-8ec6e0efe0e2'], 'standard_attr_id': 411, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:03:00Z', 'updated_at': '2026-05-27T16:03:00Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 33476}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.933466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-26791013-51e5-4bfa-9444-c469ccd1f068, col_values=(('external_ids', {'neutron:network_name': 'tempest-ServerActionsTestOtherA-1037644522-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.933901 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ddf059e7-77ce-4984-8680-578d8cdc3c78, external_ids_update=None, columns={'tag': 33476}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.933901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.934155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:57.934155 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:57.934155 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:57.934155 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:57.934155 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:57.934155 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:57.934155 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.934155 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 26791013-51e5-4bfa-9444-c469ccd1f068 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:57.934898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:57.935209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:57.935538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:57.935538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.935538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:57.935538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e2510e0f-28d1-484d-8605-6e6fbc1e01e2 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:57.964929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e2510e0f-28d1-484d-8605-6e6fbc1e01e2, resource={'id': 'e2510e0f-28d1-484d-8605-6e6fbc1e01e2', 'name': 'private', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2524c773-2910-41f1-b8dd-03d9ac809fd5', '71e0c14f-375e-43c9-bf49-7597bd311ee9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T15:57:42Z', 'updated_at': '2026-05-27T15:57:44Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 8519}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.964929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e2510e0f-28d1-484d-8605-6e6fbc1e01e2, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.965194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-96ddfe89-0ab3-4b19-b3ee-13a966e7d5c2, external_ids_update=None, columns={'tag': 8519}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:57.965194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.965343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:57.965343 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:57.965343 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:57.965343 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:57.965343 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:57.965343 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:57.965343 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.965343 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource e2510e0f-28d1-484d-8605-6e6fbc1e01e2 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:57.966040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:57.966314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:57.966594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:57.966594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:57.966594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:57.966594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource d8c882b9-3186-4faa-a11c-ec517781520a (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.000961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d8c882b9-3186-4faa-a11c-ec517781520a, resource={'id': 'd8c882b9-3186-4faa-a11c-ec517781520a', 'name': 'tempest-ServerRescueNegativeTestJSON-746125810-network', 'tenant_id': '23c706ac666441018ef25a4f6d688fdf', 'project_id': '23c706ac666441018ef25a4f6d688fdf', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['87f17570-ec8f-4941-b215-6c616bfc0ce0'], 'standard_attr_id': 688, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:06:04Z', 'updated_at': '2026-05-27T16:06:05Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 18690}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.001480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d8c882b9-3186-4faa-a11c-ec517781520a, col_values=(('external_ids', {'neutron:network_name': 'tempest-ServerRescueNegativeTestJSON-746125810-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.001480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-63e4a337-4a8e-41b7-bc88-dc828bd26a76, external_ids_update=None, columns={'tag': 18690}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.001617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.003192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.003192 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.003192 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.003192 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.003192 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.003192 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.003192 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.003192 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource d8c882b9-3186-4faa-a11c-ec517781520a (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.004842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.005490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.006433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.006433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.006433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.006433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource b2dfb7b1-f29a-41ef-9c6a-1e6d20c31be1 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.048576 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b2dfb7b1-f29a-41ef-9c6a-1e6d20c31be1, resource={'id': 'b2dfb7b1-f29a-41ef-9c6a-1e6d20c31be1', 'name': 'tempest-VolumesBackupsTest-1706551192-network', 'tenant_id': '120cf47af1d8477ba49234004da92bc2', 'project_id': '120cf47af1d8477ba49234004da92bc2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['d268fb5e-31af-4b95-b4db-a5332eb80c23'], 'standard_attr_id': 541, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:03:24Z', 'updated_at': '2026-05-27T16:03:25Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 48176}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.049003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b2dfb7b1-f29a-41ef-9c6a-1e6d20c31be1, col_values=(('external_ids', {'neutron:network_name': 'tempest-VolumesBackupsTest-1706551192-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.049438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-fec8c304-f3c5-4aa3-9b03-1a2001cbfc2c, external_ids_update=None, columns={'tag': 48176}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.049593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.050380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.050380 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.050380 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.050380 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.050380 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.050380 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.050380 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.050380 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource b2dfb7b1-f29a-41ef-9c6a-1e6d20c31be1 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.051676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.052080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.052340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.052340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.052340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.052340 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 30358808-4df7-4934-94cc-22f95bb21fa8 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.088744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-30358808-4df7-4934-94cc-22f95bb21fa8, resource={'id': '30358808-4df7-4934-94cc-22f95bb21fa8', 'name': 'tempest-AttachVolumeTestJSON-1024614630-network', 'tenant_id': '682646bd51194c46904dc0a72cbdc988', 'project_id': '682646bd51194c46904dc0a72cbdc988', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['a2fc2688-7a02-40ad-94d7-61b414c21844'], 'standard_attr_id': 590, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:03:42Z', 'updated_at': '2026-05-27T16:03:43Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 9148}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.088972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-30358808-4df7-4934-94cc-22f95bb21fa8, col_values=(('external_ids', {'neutron:network_name': 'tempest-AttachVolumeTestJSON-1024614630-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.089219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-dd357fb1-07ca-412d-9993-cc650270c24b, external_ids_update=None, columns={'tag': 9148}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.089323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.089942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.089942 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.089942 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.089942 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.089942 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.089942 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.089942 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.089942 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 30358808-4df7-4934-94cc-22f95bb21fa8 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.090682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.090944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.091262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.091262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.091262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.091262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 7825f20e-4996-4d0b-9c5b-68726ff1c557 (type: security_groups) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.124114 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Successfully bumped revision number for resource 7825f20e-4996-4d0b-9c5b-68726ff1c557 (type: security_groups) to 3
May 27 16:07:58.124114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource eb7a9a5e-286e-42e0-bfee-7e08a6a91cff (type: security_groups) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.153490 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Successfully bumped revision number for resource eb7a9a5e-286e-42e0-bfee-7e08a6a91cff (type: security_groups) to 3
May 27 16:07:58.153655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 0b7466ea-c2b3-47b3-9ec5-35a9a53ec94d (type: security_groups) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.177392 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Successfully bumped revision number for resource 0b7466ea-c2b3-47b3-9ec5-35a9a53ec94d (type: security_groups) to 3
May 27 16:07:58.177519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 71a16b6e-bd23-4abf-a692-71e291ac35bf (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.245205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, resource={'id': '71a16b6e-bd23-4abf-a692-71e291ac35bf', 'name': 'tempest-VolumesBackupsTest-1706551192-router', 'project_id': '120cf47af1d8477ba49234004da92bc2', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.33'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::16a'}], 'enable_snat': True}, 'gw_port_id': '976929cf-ee16-4c52-add4-4668895a378a', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.33', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::16a', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:03:26Z', 'updated_at': '2026-05-27T16:03:28Z', 'revision_number': 3, 'tenant_id': '120cf47af1d8477ba49234004da92bc2'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.245450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, columns={'external_ids': {'neutron:router_name': 'tempest-VolumesBackupsTest-1706551192-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.245722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "976929cf-ee16-4c52-add4-4668895a378a"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.245920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "976929cf-ee16-4c52-add4-4668895a378a"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.246057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.246664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.246664 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.246664 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.246664 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.246664 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.246664 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.246664 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.246664 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.246936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 71a16b6e-bd23-4abf-a692-71e291ac35bf. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 71a16b6e-bd23-4abf-a692-71e291ac35bf (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:07:58.247857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.248103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.248399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.248399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.248399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.248399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.248399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.248399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.248399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.248399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.248399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.248399 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 720585d3-b835-4aa8-acd5-95e640b4df8e (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.315119 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, resource={'id': '720585d3-b835-4aa8-acd5-95e640b4df8e', 'name': 'tempest-AttachVolumeTestJSON-1024614630-router', 'project_id': '682646bd51194c46904dc0a72cbdc988', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.206'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::234'}], 'enable_snat': True}, 'gw_port_id': '25939f7c-c397-475b-8c5f-260598cf1554', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.206', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::234', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:03:44Z', 'updated_at': '2026-05-27T16:03:46Z', 'revision_number': 3, 'tenant_id': '682646bd51194c46904dc0a72cbdc988'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.315318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, columns={'external_ids': {'neutron:router_name': 'tempest-AttachVolumeTestJSON-1024614630-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.315451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "25939f7c-c397-475b-8c5f-260598cf1554"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.315564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "25939f7c-c397-475b-8c5f-260598cf1554"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.315685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.316164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.316164 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.316164 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.316164 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.316164 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.316164 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.316164 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.316164 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.316434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 720585d3-b835-4aa8-acd5-95e640b4df8e. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 720585d3-b835-4aa8-acd5-95e640b4df8e (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:07:58.317187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.317444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.317697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.317697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.317697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.317697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.317697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.317697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.317697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.317697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.317697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.317697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 46e99275-40fc-40f2-bf36-87e245d6f956 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.382004 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, resource={'id': '46e99275-40fc-40f2-bf36-87e245d6f956', 'name': 'tempest-ServerRescueNegativeTestJSON-746125810-router', 'project_id': '23c706ac666441018ef25a4f6d688fdf', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.138'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::294'}], 'enable_snat': True}, 'gw_port_id': '896470d9-55ac-4bdf-9ed9-43acbefcd53f', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.138', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::294', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:06:06Z', 'updated_at': '2026-05-27T16:06:09Z', 'revision_number': 3, 'tenant_id': '23c706ac666441018ef25a4f6d688fdf'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.382285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, columns={'external_ids': {'neutron:router_name': 'tempest-ServerRescueNegativeTestJSON-746125810-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.382513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "896470d9-55ac-4bdf-9ed9-43acbefcd53f"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.382710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "896470d9-55ac-4bdf-9ed9-43acbefcd53f"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.382823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.383382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.383382 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.383382 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.383382 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.383382 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.383382 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.383382 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.383382 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.383760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 46e99275-40fc-40f2-bf36-87e245d6f956. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 46e99275-40fc-40f2-bf36-87e245d6f956 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:07:58.385203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.385606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.385989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.385989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.385989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.385989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.385989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.385989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.385989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.385989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.385989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.385989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource a94b1447-8745-4f1c-b2db-8626b23529b2 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.451326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, resource={'id': 'a94b1447-8745-4f1c-b2db-8626b23529b2', 'name': 'tempest-ServerActionsTestOtherA-1037644522-router', 'project_id': '8245a9a47a0544aa865dde98974ae162', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.70'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::1a8'}], 'enable_snat': True}, 'gw_port_id': 'b46889c3-71a3-40bd-b6fe-9ed6a5986c24', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.70', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::1a8', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:03:02Z', 'updated_at': '2026-05-27T16:03:04Z', 'revision_number': 3, 'tenant_id': '8245a9a47a0544aa865dde98974ae162'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.451557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, columns={'external_ids': {'neutron:router_name': 'tempest-ServerActionsTestOtherA-1037644522-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.451993 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "b46889c3-71a3-40bd-b6fe-9ed6a5986c24"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.451993 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "b46889c3-71a3-40bd-b6fe-9ed6a5986c24"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.451993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.452544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.452544 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.452544 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.452544 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.452544 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.452544 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.452544 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.452544 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.452866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router a94b1447-8745-4f1c-b2db-8626b23529b2. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource a94b1447-8745-4f1c-b2db-8626b23529b2 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:07:58.453917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.454313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.455177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.455177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.455177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.455177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.455177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.455177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.455177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.455177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.455177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.455177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 976929cf-ee16-4c52-add4-4668895a378a (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.586936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-976929cf-ee16-4c52-add4-4668895a378a, resource={'id': '976929cf-ee16-4c52-add4-4668895a378a', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:fe:82:b2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '71a16b6e-bd23-4abf-a692-71e291ac35bf', 'device_owner': 'network:router_gateway', 'standard_attr_id': 546, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.33'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::16a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:03:26Z', 'updated_at': '2026-05-27T16:03:29Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.587203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-976929cf-ee16-4c52-add4-4668895a378a, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::16a/64', '172.24.5.33/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.587203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=976929cf-ee16-4c52-add4-4668895a378a, lrouter_port=lrp-976929cf-ee16-4c52-add4-4668895a378a, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.587203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.588103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.588103 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.588103 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.588103 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.588103 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.588103 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.588103 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.588103 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 976929cf-ee16-4c52-add4-4668895a378a (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.589157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.589903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.590284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.590284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.590284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.590284 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 25939f7c-c397-475b-8c5f-260598cf1554 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.708140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-25939f7c-c397-475b-8c5f-260598cf1554, resource={'id': '25939f7c-c397-475b-8c5f-260598cf1554', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:da:b0:45', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '720585d3-b835-4aa8-acd5-95e640b4df8e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 595, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.206'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::234'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:03:44Z', 'updated_at': '2026-05-27T16:03:47Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.708400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-25939f7c-c397-475b-8c5f-260598cf1554, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-720585d3-b835-4aa8-acd5-95e640b4df8e'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.206/24', '2001:db8::234/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.708738 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=25939f7c-c397-475b-8c5f-260598cf1554, lrouter_port=lrp-25939f7c-c397-475b-8c5f-260598cf1554, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.708986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.709537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.709537 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.709537 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.709537 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.709537 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.709537 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.709537 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.709537 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 25939f7c-c397-475b-8c5f-260598cf1554 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.710971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.713214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.714612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.714612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.714612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.714612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 896470d9-55ac-4bdf-9ed9-43acbefcd53f (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.816897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-896470d9-55ac-4bdf-9ed9-43acbefcd53f, resource={'id': '896470d9-55ac-4bdf-9ed9-43acbefcd53f', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:e5:e0:b6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '46e99275-40fc-40f2-bf36-87e245d6f956', 'device_owner': 'network:router_gateway', 'standard_attr_id': 693, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.138'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::294'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:06:06Z', 'updated_at': '2026-05-27T16:06:09Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.819036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-896470d9-55ac-4bdf-9ed9-43acbefcd53f, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-46e99275-40fc-40f2-bf36-87e245d6f956'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::294/64', '172.24.5.138/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.819036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=896470d9-55ac-4bdf-9ed9-43acbefcd53f, lrouter_port=lrp-896470d9-55ac-4bdf-9ed9-43acbefcd53f, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.819036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.819036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.819036 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.819036 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.819036 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.819036 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.819036 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 896470d9-55ac-4bdf-9ed9-43acbefcd53f (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.820042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.822437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.823865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.823865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.823865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.823865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.823865 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource b46889c3-71a3-40bd-b6fe-9ed6a5986c24 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:07:58.948306 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-b46889c3-71a3-40bd-b6fe-9ed6a5986c24, resource={'id': 'b46889c3-71a3-40bd-b6fe-9ed6a5986c24', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:62:cc:20', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'a94b1447-8745-4f1c-b2db-8626b23529b2', 'device_owner': 'network:router_gateway', 'standard_attr_id': 416, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.70'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::1a8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:03:02Z', 'updated_at': '2026-05-27T16:03:05Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.948723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-b46889c3-71a3-40bd-b6fe-9ed6a5986c24, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-a94b1447-8745-4f1c-b2db-8626b23529b2'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::1a8/64', '172.24.5.70/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.949147 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=b46889c3-71a3-40bd-b6fe-9ed6a5986c24, lrouter_port=lrp-b46889c3-71a3-40bd-b6fe-9ed6a5986c24, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:58.949445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.950191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:58.950191 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.950191 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:58.950191 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:58.950191 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.950191 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:58.950191 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.950191 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource b46889c3-71a3-40bd-b6fe-9ed6a5986c24 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.951073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:58.951269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:58.951477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:58.951477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:58.951477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:58.951477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 010a94b3-3a2d-4d78-8d3f-1fdf9972c5f5 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:58.951477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(010a94b3-3a2d-4d78-8d3f-1fdf9972c5f5) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:58.956507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 01d9d45b-080a-455f-8e17-c0ca140c22d4 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:58.956776 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(01d9d45b-080a-455f-8e17-c0ca140c22d4) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:58.960816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 06877545-428d-48dc-b0b8-938891fd89d1 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:58.961046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(06877545-428d-48dc-b0b8-938891fd89d1) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:58.965317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 38196e44-6d13-4447-a839-1e0c6d26bdbf (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:58.965533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(38196e44-6d13-4447-a839-1e0c6d26bdbf) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:58.970416 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 48f3f4fd-e6bd-4c52-9902-e1b594da530d (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:58.970597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(48f3f4fd-e6bd-4c52-9902-e1b594da530d) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:58.975464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 4dac21e3-9416-4fa1-b764-fc796a89687e (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:58.975775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(4dac21e3-9416-4fa1-b764-fc796a89687e) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:58.980708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 8306982f-e280-4988-aad3-3d0e363d80d3 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:58.981024 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(8306982f-e280-4988-aad3-3d0e363d80d3) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:58.987552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 917bf073-0dcd-44b1-962e-ce1cf1407e9d (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:58.987966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(917bf073-0dcd-44b1-962e-ce1cf1407e9d) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:58.993775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 927f2797-254e-4745-ac64-c4a87d401f63 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:58.994079 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(927f2797-254e-4745-ac64-c4a87d401f63) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:58.998369 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 9c6735ec-6bc1-4ae5-ac63-5960d30a4501 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:58.998706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(9c6735ec-6bc1-4ae5-ac63-5960d30a4501) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:59.004211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource a2660100-9cf9-436c-8a0a-61b227f3810b (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:59.004467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(a2660100-9cf9-436c-8a0a-61b227f3810b) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:59.008552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource d24ebe2d-8475-4f0a-a6f5-a91d2585a351 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:59.008792 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(d24ebe2d-8475-4f0a-a6f5-a91d2585a351) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:59.013092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource d4282b9d-e650-4dbc-b9b6-7eca13141b9b (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:59.013288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(d4282b9d-e650-4dbc-b9b6-7eca13141b9b) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:59.017203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource d79d51fe-3a72-464e-a132-71c7f7bdddf9 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:59.017405 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(d79d51fe-3a72-464e-a132-71c7f7bdddf9) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:59.021299 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e8319168-cbca-43c8-bd05-ed617c7c91dc (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:59.021496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(e8319168-cbca-43c8-bd05-ed617c7c91dc) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:59.025451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource ec395440-0360-4fed-8b8a-dcd740d28cc4 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:07:59.025646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(ec395440-0360-4fed-8b8a-dcd740d28cc4) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:07:59.029501 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_for_inconsistencies finished in 1.214 seconds
May 27 16:07:59.029691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:59.029771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:59.030560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:59.064755 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.064755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.065329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:59.065329 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.065329 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:59.065329 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:59.065329 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.065329 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:59.065329 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.065329 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.066882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.068248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:59.068988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:59.077494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.078296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:59.078296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:59.078296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.078296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:59.078296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.078296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:59.125867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:59.126505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:59.126505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:59.127882 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.128095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.128537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.128537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.128537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.128537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.129226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.129226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.129226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.129226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.129810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:59.129810 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.129810 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:59.129810 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:59.129810 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.129810 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:59.129810 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.129810 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.130998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.133191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:59.135791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:59.137815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.140412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:59.140412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:59.140412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.140412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:59.140412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.140412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:59.140412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:59.140412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:59.140412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:59.140412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:59.142236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.145572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:59.148324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:59.150375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:59.152547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.153789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.155137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:59.365344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:59.366010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:59.366010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:59.366096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:59.366096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.367345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.367345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:59.367345 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.367345 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:59.367345 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:59.367345 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.367345 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:59.367345 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.367345 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:07:59.368090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:59.369550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:59.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:59.372398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:07:59.373470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.374783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:07:59.397040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:07:59.397295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:07:59.397379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:07:59.397526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:07:59.399134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.399134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:07:59.399134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.399284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:07:59.399284 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.399284 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:07:59.399284 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:07:59.399284 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.399284 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:07:59.399284 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.399284 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.399905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.400299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:07:59.400676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:07:59.401072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:07:59.401583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:07:59.401583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:07:59.401583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:07:59.401583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:07:59.401583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:07:59.401583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:04.029958 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:04.030538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:04.030538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:04.030936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:04.066406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.066406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.066643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:04.066643 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.066643 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:04.066643 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:04.066643 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.066643 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:04.066643 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.066643 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.067448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.067865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:04.068212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:04.068638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.069155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:04.069155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:04.069155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.069155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:04.069155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.069155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:04.127867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:04.128307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:04.128457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:04.131381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.131381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.131381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.131381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.131381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.131590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.131590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.131590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.131590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.131590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.131933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:04.131933 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.131933 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:04.131933 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:04.131933 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.131933 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:04.131933 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.131933 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.132963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.133289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:04.133645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.133645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:04.134077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:04.134487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.134989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:04.134989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.134989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:04.136060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:04.136330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:04.136437 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:04.136537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:04.137463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.137463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.137810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:04.137810 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.137810 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:04.137810 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:04.137810 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.137810 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:04.137810 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.137810 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:04.138693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:04.139412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.139412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:04.139412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.139412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:04.139412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:04.139412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.139412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:04.139412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.139412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.139885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:04.140509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:04.141176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.142720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.143955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:04.366026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:04.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:04.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:04.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:04.366943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.367186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.367602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:04.367602 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.367602 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:04.367602 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:04.367602 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.367602 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:04.367602 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.367602 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:04.370762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:04.371066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:04.371362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:04.371593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:04.372049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.372418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:04.398488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:04.398927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:04.399076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:04.399266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:04.399593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.399826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:04.400076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.400644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:04.400644 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.400644 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:04.400644 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:04.400644 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.400644 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:04.400644 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.400644 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.401432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.401694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:04.401895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.401895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:04.402231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:04.402509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:04.402827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:04.402827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:04.402827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:09.030406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:09.030808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:09.030808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:09.031474 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:09.077696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.077910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.078484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:09.078484 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.078484 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:09.078484 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:09.078484 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.078484 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:09.078484 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.078484 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.079168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.079689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:09.080076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:09.080549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.081215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:09.081215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:09.081215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.081215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:09.081215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.081215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:09.128290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:09.130655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:09.130655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:09.130655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.130655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.130655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.130886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.130886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.130886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.130886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.130886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.131076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.131076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.131076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:09.131076 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.131076 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:09.131076 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:09.131076 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.131076 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:09.131076 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.131076 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.131668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.132051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:09.132401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.132401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:09.132553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:09.132995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.133512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:09.133512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.133512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:09.138644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:09.138644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:09.138644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:09.138644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:09.138644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.138644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.138875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:09.139355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.139759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.140081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:09.140429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:09.140816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:09.141253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.141253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:09.141253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.141253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:09.141253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:09.141253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.141253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:09.141253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.141253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:09.367701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:09.367916 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:09.368008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:09.368097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:09.368442 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.368701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.369278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:09.369278 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.369278 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:09.369278 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:09.369278 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.369278 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:09.369278 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.369278 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:09.370094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:09.370360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.370360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:09.370360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.370360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:09.370360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:09.370360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.370360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:09.370360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.370360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.371009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:09.371216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:09.371578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.371578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:09.371578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.371578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.372038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:09.372334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:09.372666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.372666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:09.372666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.372666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:09.398909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:09.399165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:09.399264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:09.399417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:09.399916 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.400197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:09.400487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.401135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:09.401135 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.401135 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:09.401135 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:09.401135 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.401135 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:09.401135 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.401135 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.402232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.402517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:09.402766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.402766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:09.407672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:09.408645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:09.409260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:09.409260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:09.409260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:14.031017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:14.031682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:14.031682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:14.032145 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:14.101001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.101364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.102036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:14.102036 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.102036 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:14.102036 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:14.102036 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.102036 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:14.102036 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.102036 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.071 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.103111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.103387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:14.103612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.103612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:14.104262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:14.104574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.104909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:14.104909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.104909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:14.128823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:14.129264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:14.129328 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:14.130598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.130822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.130965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.131137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.131290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.131446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.132331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.132331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.132331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a94b1447-8745-4f1c-b2db-8626b23529b2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.132331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.132715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:14.132715 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.132715 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:14.132715 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:14.132715 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.132715 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:14.132715 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.132715 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.133581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.133864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:14.134079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.134079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:14.134726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:14.135006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.135349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:14.135349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.135349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:14.136702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:14.136980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:14.137157 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:14.137202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:14.137645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.137840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.138316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:14.138316 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.138316 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:14.138316 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:14.138316 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.138316 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:14.138316 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.138316 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:14.138867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:14.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:14.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:14.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:14.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:14.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.139373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.139662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:14.140029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:14.140430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.140841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.141351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:14.368458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:14.368884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:14.369008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:14.369193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:14.369727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.370298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.370651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:14.370651 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.370651 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:14.370651 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:14.370651 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.370651 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:14.370651 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.370651 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:14.371511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.371916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.373308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:14.373697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:14.376089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:14.376564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.377087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:14.399394 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:14.400216 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:14.400216 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:14.400216 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:14.400363 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.400874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:14.400931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.401514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:14.401514 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.401514 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:14.401514 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:14.401514 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.401514 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:14.401514 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.401514 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.402555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.403583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:14.404016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:14.404517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:14.405126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:14.405126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:14.405126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:14.405126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:14.405126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:14.405126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:19.031881 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:19.032335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:19.032493 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:19.033342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:19.069248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.069248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.069248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:19.069248 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.069248 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:19.069248 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:19.069248 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.069248 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:19.069248 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.069248 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.069649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.069899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:19.070166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.070166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:19.070461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:19.070824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.071217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:19.071217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.071217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:19.129360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:19.129582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:19.129754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:19.130642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.130803 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.130928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.131051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.131168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.131320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.131426 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.131519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.131641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.132102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:19.132102 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.132102 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:19.132102 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:19.132102 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.132102 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:19.132102 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.132102 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.133025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.133411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:19.133762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.133762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:19.134066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:19.134364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.134724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:19.134724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.134724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:19.137147 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:19.137849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:19.137849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:19.137988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:19.138650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.138650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.139077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:19.139077 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.139077 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:19.139077 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:19.139077 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.139077 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:19.139077 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.139077 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:19.139730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:19.140013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.140013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:19.140013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.140013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:19.140013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:19.140013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.140013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:19.140013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.140013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.140371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:19.140632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:19.140868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.140868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:19.140868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.140868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:19.141318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.141596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:19.141925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.141925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:19.141925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:19.141925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.141925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:19.141925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.141925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:19.368884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:19.369254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:19.369396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:19.369568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:19.369937 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.370176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.370640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:19.370640 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.370640 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:19.370640 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:19.370640 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.370640 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:19.370640 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.370640 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:19.371315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:19.371821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.371821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:19.371821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.371821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:19.371821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:19.371821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.371821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:19.371821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.371821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.372060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:19.372389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:19.372769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.372769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:19.372769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.372769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.373281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:19.373772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:19.374274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.374274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:19.374274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.374274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:19.399877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:19.400283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:19.400374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:19.400517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:19.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.401263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:19.401494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.401991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:19.401991 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.401991 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:19.401991 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:19.401991 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.401991 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:19.401991 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.401991 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.402689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:19.403168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.403168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:19.403613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:19.403938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:19.404272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:19.404272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:19.404272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:24.032319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:24.033262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:24.034856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:24.034856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:24.073411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.073574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.074045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:24.074045 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.074045 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:24.074045 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:24.074045 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.074045 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:24.074045 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.074045 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.075116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.075465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:24.075741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.075741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:24.076238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:24.076597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.076978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:24.076978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.076978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:24.129766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:24.130148 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:24.130271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:24.131095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.131246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.131385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.131496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.131605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.131752 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.131851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.131944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.132066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.132575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:24.132575 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.132575 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:24.132575 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:24.132575 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.132575 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:24.132575 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.132575 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.133291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.133720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:24.133950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.133950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:24.134610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:24.135136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.135596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:24.135596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.135596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:24.137899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:24.138152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:24.138254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:24.138351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:24.138650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.138873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.139212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:24.139212 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.139212 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:24.139212 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:24.139212 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.139212 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:24.139212 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.139212 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:24.139724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:24.140124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.140124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:24.140124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.140124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:24.140124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:24.140124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.140124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:24.140124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.140124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.140363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:24.140681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:24.141021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.141397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.141920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:24.369321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:24.369715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:24.369853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:24.369992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:24.370396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.370701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.371791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:24.371791 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.371791 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:24.371791 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:24.371791 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.371791 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:24.371791 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.371791 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:24.372581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:24.373039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.373039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:24.373039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.373039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:24.373039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:24.373039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.373039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:24.373039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.373039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.373590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:24.373945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:24.374378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.374378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:24.374378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.374378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.374757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:24.375112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:24.375571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.375571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:24.375571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.375571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:24.400310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:24.400602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:24.400759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:24.400940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:24.401328 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.401640 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:24.401919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.402525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:24.402525 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.402525 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:24.402525 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:24.402525 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.402525 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:24.402525 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.402525 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.403293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.403688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:24.404048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.404048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:24.404355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:24.404855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:24.405476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:24.405476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:24.405476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:29.033838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:29.033838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:29.033838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:29.034427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:29.088183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.088670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.089167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:29.089167 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.089167 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:29.089167 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:29.089167 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.089167 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:29.089167 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.089167 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.056 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.091291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.091880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:29.092262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:29.092693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.093218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:29.093218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:29.093218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.093218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:29.093218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.093218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:29.131306 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:29.131380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:29.131553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:29.134011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.134229 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.134369 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.134583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.134674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.134760 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.134843 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.134984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.135042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.136683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:29.136683 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.136683 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:29.136683 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:29.136683 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.136683 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:29.136683 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.136683 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.137576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.137973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:29.138204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:29.138496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.138879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:29.138879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:29.138879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.138879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:29.138879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.138879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:29.138879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:29.138879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:29.138879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:29.138879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:29.139601 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.139915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.140507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:29.140507 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.140507 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:29.140507 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:29.140507 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.140507 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:29.140507 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.140507 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:29.141068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:29.141315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.141315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:29.141315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.141315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:29.141315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:29.141315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.141315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:29.141315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.141315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.141766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:29.141960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:29.142223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.142223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:29.142223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.142223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:29.142531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.142828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:29.143091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.143091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:29.143091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:29.143091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.143091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:29.143091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.143091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:29.369801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:29.370162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:29.370352 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:29.370488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:29.371091 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.371296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.371790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:29.371790 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.371790 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:29.371790 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:29.371790 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.371790 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:29.371790 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.371790 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:29.372499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:29.373060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.373060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:29.373060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.373060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:29.373060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:29.373060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.373060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:29.373060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.373060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.373303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:29.373728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:29.374148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:29.374497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.375034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:29.400810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:29.401191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:29.401405 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:29.401567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:29.401972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.402282 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:29.402587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.403158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:29.403158 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.403158 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:29.403158 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:29.403158 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.403158 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:29.403158 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.403158 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.403945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.404716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:29.405079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:29.405408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:29.405812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:29.405812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:29.405812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:29.405812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:29.405812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:29.405812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:34.034895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:34.035421 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:34.035539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:34.036179 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:34.087772 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.088003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.088589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:34.088589 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.088589 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:34.088589 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:34.088589 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.088589 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:34.088589 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.088589 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.089571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.089841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:34.090051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.090051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:34.090970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:34.093496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.094108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:34.094108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.094108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:34.131790 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:34.131790 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:34.131932 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:34.134197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.134399 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.134571 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.134712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.134823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.134922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.135052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.135140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.135285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.136023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:34.136023 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.136023 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:34.136023 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:34.136023 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.136023 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:34.136023 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.136023 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.137601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.138011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:34.138228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:34.138510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.138899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:34.138899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:34.138899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.138899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:34.138899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.138899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:34.138899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:34.139027 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:34.140373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:34.140373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:34.143152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.143304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.145947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:34.145947 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.145947 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:34.145947 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:34.145947 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.145947 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:34.145947 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.145947 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:34.146792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:34.147039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.147039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:34.147039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.147039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:34.147039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:34.147039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.147039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:34.147039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.147039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.147699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:34.148006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:34.148250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.148250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:34.148250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.148250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:34.148902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.149388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:34.149789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.149789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:34.149789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:34.149789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.149789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:34.149789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.149789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:34.370334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:34.370682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:34.370850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:34.370935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:34.371700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.371772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.372315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:34.372315 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.372315 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:34.372315 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:34.372315 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.372315 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:34.372315 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.372315 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:34.373354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:34.373670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.373670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:34.373670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.373670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:34.373670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:34.373670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.373670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:34.373670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.373670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.374414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:34.374642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:34.374876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.374876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:34.374876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.374876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.375732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:34.376050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:34.376419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.376419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:34.376419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.376419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:34.401308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:34.401886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:34.401886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:34.401886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:34.402424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.402732 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:34.402976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.403798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:34.403798 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.403798 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:34.403798 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:34.403798 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.403798 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:34.403798 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.403798 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.404896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.405297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:34.405717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.405717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:34.406087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:34.406726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:34.407293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:34.407293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:34.407293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:39.036516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:39.036996 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:39.037196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:39.038113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:39.122611 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.122611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.123395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:39.123395 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.123395 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:39.123395 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:39.123395 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.123395 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:39.123395 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.123395 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.086 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.124332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.124722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:39.125021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.125021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:39.125389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:39.125723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.126103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:39.126103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.126103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:39.131934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:39.132222 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:39.132380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:39.133992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.134165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.134314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.134459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.134577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.134732 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.134848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.134985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.135084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-720585d3-b835-4aa8-acd5-95e640b4df8e, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.135195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.135724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:39.135724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.135724 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:39.135724 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:39.135724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.135724 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:39.135724 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.135724 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.136679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.137040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:39.137530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.137530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:39.137689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:39.138016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.138354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:39.138354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.138354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:39.139956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:39.139956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:39.139956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:39.139956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:39.141301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.141430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.142143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:39.142143 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.142143 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:39.142143 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:39.142143 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.142143 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:39.142143 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.142143 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:39.142816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:39.143343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.143343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:39.143343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.143343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:39.143343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:39.143343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.143343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:39.143343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.143343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.143647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:39.144675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:39.144929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.145163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.145883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:39.370799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:39.371693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:39.371814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:39.372231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:39.374603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.374961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.375985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:39.375985 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.375985 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:39.375985 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:39.375985 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.375985 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:39.375985 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.375985 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:39.376481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:39.376848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.376848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:39.376848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.376848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:39.376848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:39.376848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.376848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:39.376848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.376848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.379487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:39.379801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:39.380134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.380134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:39.380134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.380134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.383714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:39.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:39.385549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.385549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:39.385549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.385549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:39.401680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:39.401957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:39.402059 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:39.402179 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:39.402503 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.402702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:39.402917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.403328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:39.403328 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.403328 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:39.403328 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:39.403328 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.403328 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:39.403328 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.403328 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.404297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:39.404696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:39.405081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:39.405551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:39.405551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:39.405551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:39.405551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:39.405551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:39.405551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:44.036967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:44.038189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:44.038299 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:44.039058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:44.097679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.098072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.099734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:44.099734 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.099734 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:44.099734 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:44.099734 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.099734 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:44.099734 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.099734 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.100881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.101188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:44.101485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.101485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:44.101952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:44.102320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.102788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:44.102788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.102788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:44.133041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:44.133041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:44.133041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:44.134037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.134266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.134360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.134944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.134944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.135144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.135444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.136252 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.136252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.138945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:44.138945 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.138945 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:44.138945 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:44.138945 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.138945 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:44.138945 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.138945 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.142506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.142827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:44.143114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.143114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:44.143114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:44.143563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:44.143960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.144417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:44.144417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.144417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:44.144417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:44.144417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:44.144417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:44.144953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.145227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.145893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:44.145893 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.145893 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:44.145893 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:44.145893 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.145893 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:44.145893 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.145893 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:44.146645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:44.146988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.146988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:44.146988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.146988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:44.146988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:44.146988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.146988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:44.146988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.146988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.147428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:44.147753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:44.148064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.148064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:44.148064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.148064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:44.148812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.149184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:44.149590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.149590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:44.149590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:44.149590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.149590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:44.149590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.149590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:44.372759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:44.372759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:44.372759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:44.372759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:44.373259 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.373616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.375655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.376053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.377555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:44.377763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:44.378012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.378012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:44.378012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.378012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.379815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:44.380441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:44.380835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.380835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:44.380835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.380835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:44.402258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:44.403176 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:44.403176 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:44.403176 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:44.413152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.413403 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:44.413569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:44.416772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:44.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:44.417328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:44.417584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:44.418656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:44.418656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:44.418656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:44.418656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:44.418656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:44.418656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:44.418656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:44.418656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:44.418656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:44.418656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:49.038257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:49.038569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:49.038753 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:49.039327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:49.078476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.078805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.079465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:49.079465 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.079465 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:49.079465 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:49.079465 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.079465 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:49.079465 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.079465 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.080475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.080734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:49.080995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.080995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:49.081479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:49.081774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.082061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:49.082061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.082061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:49.132972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:49.133273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:49.133429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:49.135481 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.136102 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.136291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.136991 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.137550 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.139337 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-71a16b6e-bd23-4abf-a692-71e291ac35bf, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.140207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.140776 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.141391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.143288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:49.143288 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.143288 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:49.143288 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:49.143288 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.143288 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:49.143288 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.143288 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.144578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.145059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:49.145394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:49.145937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.146457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:49.146457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:49.146457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.146457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:49.146457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.146457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:49.146457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:49.146457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:49.146457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:49.146457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:49.146775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.146775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.147031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:49.147031 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.147031 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:49.147031 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:49.147031 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.147031 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:49.147031 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.147031 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:49.147929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:49.148260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.148561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.148872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:49.149267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.149757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:49.150182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.150182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:49.150182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:49.150182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.150182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:49.150182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.150182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:49.372327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:49.372610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:49.372672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:49.372784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:49.373225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.373530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.374111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:49.374111 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.374111 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:49.374111 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:49.374111 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.374111 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:49.374111 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.374111 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:49.375541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:49.375832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.375832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:49.375832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.375832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:49.375832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:49.375832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.375832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:49.375832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.375832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.376022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:49.376224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:49.376613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.376613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:49.376613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.376613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.379453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:49.381283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:49.383323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.383323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:49.383323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.383323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:49.402842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:49.402842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:49.402842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:49.402842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:49.402842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.402842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:49.407249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.407249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:49.407249 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.407249 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:49.407249 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:49.407249 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.407249 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:49.407249 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.407249 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.408163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.408375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:49.408600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.408600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:49.409393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:49.409720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:49.410040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:49.410040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:49.410040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:54.039870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:54.039870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:54.041039 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:54.043019 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:54.046566 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=876,neutron:UPDATE=3,neutron:DELETE=16 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:08:54.106333 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.106598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.107210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:54.107210 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.107210 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:54.107210 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:54.107210 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.107210 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:54.107210 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.107210 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.067 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.108236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.108670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:54.109192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.109192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:54.109255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:54.109752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.110289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:54.110289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.110289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:54.133490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:54.134238 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:54.134238 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:54.138864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.138864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.138864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.138864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.138864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.139365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.139365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.139365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.141486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:54.141486 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.141486 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:54.141486 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:54.141486 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.141486 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:54.141486 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.141486 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.144898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.146056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:54.146399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:54.146869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.147348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:54.147348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:54.147348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.147348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:54.147348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.147348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:54.147348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:54.149734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:54.149734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:54.149734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:54.149734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.149734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.149734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:54.149734 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.149734 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:54.149734 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:54.150318 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.150318 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:54.150318 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.150318 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:54.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:54.152946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.153341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.153719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:54.155226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.155652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:54.156060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.156060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:54.156060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:54.156060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.156060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:54.156060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.156060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:54.373293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:54.374052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:54.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:54.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:54.375702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.375914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.377298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:54.377298 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.377298 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:54.377298 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:54.377298 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.377298 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:54.377298 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.377298 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:54.378721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:54.379258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.379258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:54.379258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.379258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:54.379258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:54.379258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.379258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:54.379258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.379258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.379557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:54.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:54.380236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.380236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:54.380236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.380236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.381704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:54.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:54.382652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.382652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:54.382652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.382652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:54.403825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:54.403825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:54.403825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:54.403825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:54.404526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.405263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:54.405263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.406187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:54.406187 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.406187 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:54.406187 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:54.406187 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.406187 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:54.406187 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.406187 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.407402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:54.408970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:54.409367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:54.409974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:54.409974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:54.409974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:54.409974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:54.409974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:54.409974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:59.039662 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:59.041051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:59.041051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:59.041051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:59.125143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.125311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.127307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:59.127307 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.127307 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:59.127307 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:59.127307 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.127307 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:59.127307 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.127307 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.086 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.127607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.127861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:59.128151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.128151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:59.128711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:59.129041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.129426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:59.129426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.129426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:59.133925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:59.134160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:59.134358 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:59.136218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.136463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.136667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.136796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.136984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.137124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.138014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.138530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:59.138530 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.138530 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:59.138530 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:59.138530 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.138530 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:59.138530 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.138530 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.139443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.139989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:59.140448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.140448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:59.140702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:59.142781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.143358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:59.143358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.143358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:59.150681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:59.150681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:59.150681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:59.150681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:08:59.150681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.150681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.151000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:59.151522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.152016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.152411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:59.152819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:59.153300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:59.154673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.154673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:59.154673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.154673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:59.154673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:59.154673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.154673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:59.154673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.154673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:59.373926 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:59.377817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:59.377817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:59.377817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:59.378124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.379155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.379155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:59.379155 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.379155 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:59.379155 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:59.379155 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.379155 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:59.379155 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.379155 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:08:59.379586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:08:59.381166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.381166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:08:59.381166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.381166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:08:59.381166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:08:59.381166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.381166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:08:59.381166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.381166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.383454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:59.383908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:59.385040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:59.386765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.387349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:08:59.403674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:08:59.404181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:08:59.404291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:08:59.404440 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:08:59.404918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.406172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:08:59.407538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.407538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:08:59.407538 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.407538 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:08:59.407538 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:08:59.407538 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.407538 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:08:59.407538 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.407538 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.408640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:08:59.409953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:08:59.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:08:59.410521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:08:59.410521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:08:59.410521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:08:59.410521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:08:59.410521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:08:59.410521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:04.043485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:04.044834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:04.044834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:04.046503 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:04.112200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.112482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.116054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:04.116054 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.116054 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:04.116054 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:04.116054 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.116054 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:04.116054 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.116054 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.072 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.118559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.118967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:04.119368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:04.119943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.120446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:04.120446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:04.120446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.120446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:04.120446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.120446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:04.134483 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:04.135032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:04.135183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:04.137644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.137644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.137644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.137960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.137960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.138124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.138262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.138414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.139678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:04.139678 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.139678 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:04.139678 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:04.139678 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.139678 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:04.139678 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.139678 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.140246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.140552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:04.140873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.140873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:04.141093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:04.141469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.141957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:04.141957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.141957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:04.148769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:04.148935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:04.149054 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:04.149178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:04.151774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.152844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.154295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:04.154295 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.154295 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:04.154295 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:04.154295 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.154295 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:04.154295 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.154295 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:04.155036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:04.155330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.155330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:04.155330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.155330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:04.155330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:04.155330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.155330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:04.155330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.155330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.156792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:04.157029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:04.157405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.157405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:04.157405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.157405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:04.158007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.158350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:04.158821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.158821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:04.158821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:04.158821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.158821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:04.158821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.158821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:04.376272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:04.376738 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:04.376990 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:04.377217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:04.377667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.378025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.378554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:04.378554 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.378554 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:04.378554 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:04.378554 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.378554 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:04.378554 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.378554 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:04.380976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:04.381258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:04.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:04.381710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:04.381907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.382204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:04.404385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:04.404884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:04.405123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:04.405433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:04.405971 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.406440 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:04.406945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:04.409936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:04.411027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:04.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:04.412540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:04.413303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:04.413303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:04.413303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:04.413303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:04.413303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:04.413303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:04.413303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:04.413303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:04.413303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:04.413303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:09.044392 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:09.044879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:09.044962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:09.045933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:09.094488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.094488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.096866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:09.096866 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.096866 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:09.096866 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:09.096866 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.096866 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:09.096866 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.096866 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.097842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.098210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:09.098639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.098639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:09.098917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:09.099305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.099946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:09.099946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.099946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:09.137279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:09.137742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:09.137964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:09.140127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.140127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.140127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.140127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.140127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.140358 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.140599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.140981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.142219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:09.142219 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.142219 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:09.142219 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:09.142219 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.142219 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:09.142219 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.142219 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.143392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.144952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:09.145247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:09.145678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.146454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:09.146454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:09.146454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.146454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:09.146454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.146454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:09.149013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:09.149635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:09.149696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:09.149829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:09.150883 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.150883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.151076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:09.151076 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.151076 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:09.151076 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:09.151076 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.151076 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:09.151076 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.151076 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:09.151821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:09.152220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.152220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:09.152220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.152220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:09.152220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:09.152220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.152220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:09.152220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.152220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.152458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:09.152814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:09.153136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.153136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:09.153136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.153136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:09.153234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.153591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:09.154007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.154007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:09.154007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:09.154007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.154007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:09.154007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.154007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:09.376825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:09.377051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:09.377149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:09.377270 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:09.377823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.378188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.378923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:09.378923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.378923 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:09.378923 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:09.378923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.378923 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:09.378923 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.378923 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:09.379676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:09.380078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.380078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:09.380078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.380078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:09.380078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:09.380078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.380078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:09.380078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.380078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.380494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:09.380867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:09.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:09.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.381224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.381338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:09.381786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:09.382269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.382269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:09.382269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.382269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:09.404976 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:09.405345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:09.405462 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:09.405685 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:09.406831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.406831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:09.406831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.406964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:09.406964 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.406964 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:09.406964 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:09.406964 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.406964 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:09.406964 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.406964 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.407764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.408270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:09.408715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.408715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:09.408819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:09.409298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:09.409727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:09.409727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:09.409727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:14.045014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:14.047121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:14.047783 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:14.056730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:14.139086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:14.154661 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:14.194296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.195139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.149 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:14.198685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:14.199081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:14.199316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:14.199560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:14.199944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.200212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.200212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.200212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.200278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.200842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.200842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.200842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.201181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:14.201181 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.201181 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:14.201181 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:14.201181 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.201181 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:14.201181 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.201181 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.203268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.203695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:14.204099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:14.204373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.204980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:14.204980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:14.204980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.204980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:14.204980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.204980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:14.204980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:14.204980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:14.204980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:14.204980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.205188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.207060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:14.207060 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.207060 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:14.207060 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:14.207060 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.207060 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:14.207060 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.207060 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:14.207592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:14.208231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.208231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:14.208231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.208231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:14.208231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:14.208231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.208231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:14.208231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.208231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.214376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:14.214747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:14.214988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.214988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:14.214988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.214988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:14.220963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.221533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:14.222120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.222120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:14.222120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:14.222120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.222120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:14.222120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.222120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:14.377447 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:14.379964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:14.380110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:14.380285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:14.386361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.388549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.388867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:14.388867 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.388867 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:14.388867 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:14.388867 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.388867 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:14.388867 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.388867 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:14.389397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:14.389944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.389944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:14.389944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.389944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:14.389944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:14.389944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.389944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:14.389944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.389944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.390395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:14.390804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:14.391247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.391247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:14.391247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.391247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.397928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:14.399491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:14.401756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.401756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:14.401756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.401756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:14.406445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:14.406846 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:14.407016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:14.407198 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:14.407773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.408162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:14.408551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.411354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:14.411354 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.411354 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:14.411354 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:14.411354 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.411354 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:14.411354 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.411354 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.419112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:14.419354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.419354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:14.426046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:14.426644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:14.427207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:14.427207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:14.427207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:19.046577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:19.047003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:19.047003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:19.047822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:19.109245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.109523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.110301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:19.110301 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.110301 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:19.110301 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:19.110301 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.110301 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:19.110301 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.110301 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.114189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.114674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:19.115121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:19.115557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.116652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:19.116652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:19.116652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.116652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:19.116652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.116652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:19.199032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:19.199389 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:19.199542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:19.200661 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.200793 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.200880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bad0d8d7-5330-46cc-afa5-c9dd92abd362, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.201411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.201411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.201411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.201411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.201699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.202116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:19.202116 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.202116 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:19.202116 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:19.202116 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.202116 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:19.202116 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.202116 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.203101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.203868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:19.204174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.204174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:19.204358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:19.204882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.206026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:19.206026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.206026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:19.206026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:19.206026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:19.206026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:19.206026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:19.206026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.206218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.206826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:19.206826 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.206826 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:19.206826 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:19.206826 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.206826 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:19.206826 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.206826 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:19.207920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:19.208245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.208245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:19.208245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.208245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:19.208245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:19.208245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.208245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:19.208245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.208245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.208496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:19.208835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:19.209165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.209165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:19.209165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.209165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:19.209355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.210026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:19.210430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.210430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:19.210430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:19.210430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.210430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:19.210430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.210430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:19.379956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:19.380166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:19.380252 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:19.380384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:19.380984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.381231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.381958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:19.381958 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.381958 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:19.381958 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:19.381958 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.381958 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:19.381958 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.381958 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:19.382645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:19.383198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.383198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:19.383198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.383198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:19.383198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:19.383198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.383198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:19.383198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.383198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.383426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:19.384035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:19.384680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:19.385213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.385947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:19.406929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:19.407248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:19.407387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:19.407599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:19.408074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.408312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:19.408501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.409124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:19.409124 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.409124 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:19.409124 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:19.409124 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.409124 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:19.409124 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.409124 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.410223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.410558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:19.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:19.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:19.411818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:19.412265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:19.412265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:19.412265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:24.049877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:24.050284 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:24.050284 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:24.050794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:24.114049 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.114220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.065 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:24.117202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:24.117800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:24.118151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:24.118455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:24.119000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.119000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:24.119000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.119000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:24.119000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:24.119000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.119000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:24.119000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.119000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:24.200437 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:24.200954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:24.201161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:24.202359 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.202638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.202839 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.202981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.203130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.203239 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.203356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:24.205399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:24.205817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.206219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:24.206219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:24.206219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.206219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:24.206219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.206219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:24.207415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:24.207947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.208419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:24.208419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.208419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:24.208419 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:24.208894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:24.209378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:24.209378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:24.212526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.212726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:24.217052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.217427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.217787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:24.218119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.218119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:24.218721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.219193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:24.220265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.220265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:24.220265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:24.220265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.220265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:24.220265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.220265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:24.380933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:24.380933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:24.381096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:24.381234 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:24.381687 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.381926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.382446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:24.382446 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.382446 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:24.382446 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:24.382446 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.382446 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:24.382446 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.382446 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:24.383425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:24.383701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.383701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:24.383701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.383701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:24.383701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:24.383701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.383701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:24.383701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.383701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.384434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:24.384681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:24.384916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.384916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:24.384916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.384916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.386400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:24.386847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:24.387135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.387135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:24.387135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.387135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:24.408101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:24.408832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:24.408832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:24.408832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:24.410361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:24.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.412420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:24.412420 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.412420 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:24.412420 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:24.412420 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.412420 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:24.412420 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.412420 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.413668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.414141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:24.414588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.414588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:24.416052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:24.416474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:24.417043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:24.417043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:24.417043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:29.050338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:29.051237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:29.051237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:29.052692 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:29.092850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.093054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.093616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:29.093616 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.093616 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:29.093616 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:29.093616 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.093616 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:29.093616 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.093616 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.094931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.095366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:29.095965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:29.096384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.096901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:29.096901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:29.096901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.096901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:29.096901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.096901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:29.201065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:29.201295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:29.201445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:29.203358 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.203599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-f4cf7e65-f890-4435-bbad-4d34f2cdf1c2, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.203872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.204058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.204210 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.204490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-fd6ecbd6-82fa-48aa-bcd6-1b32c718050c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.204490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.205235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:29.205235 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.205235 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:29.205235 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:29.205235 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.205235 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:29.205235 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.205235 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.206299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.207012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:29.207323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.207323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:29.207384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:29.207940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.208341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:29.208341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.208341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:29.209106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:29.209493 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:29.209676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:29.209803 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:29.210972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.212451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.213082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:29.213082 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.213082 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:29.213082 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:29.213082 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.213082 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:29.213082 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.213082 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:29.213815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:29.214269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.214269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:29.214269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.214269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:29.214269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:29.214269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.214269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:29.214269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.214269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.214531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:29.214995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:29.215345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.215345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:29.215345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.215345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:29.215437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.215923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:29.216398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.216398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:29.216398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:29.216398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.216398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:29.216398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.216398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:29.381000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:29.381312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:29.381431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:29.381604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:29.382235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.383243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.385087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:29.385087 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.385087 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:29.385087 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:29.385087 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.385087 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:29.385087 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.385087 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:29.385982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:29.386385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.386385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:29.386385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.386385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:29.386385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:29.386385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.386385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:29.386385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.386385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.387173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:29.387596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:29.387968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.387968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:29.387968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.387968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.388409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:29.388957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:29.389436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.389436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:29.389436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.389436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:29.408658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:29.408982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:29.409163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:29.409282 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:29.409751 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.409980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:29.410212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.410735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:29.410735 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.410735 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:29.410735 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:29.410735 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.410735 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:29.410735 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.410735 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.411458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.411840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:29.412143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.412143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:29.413239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:29.413755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:29.414055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:29.414055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:29.414055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:34.051044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:34.052838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:34.053716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:34.056122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:34.100791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.101003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.102142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:34.102142 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.102142 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:34.102142 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:34.102142 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.102142 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:34.102142 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.102142 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.103199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.103481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:34.103838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.103838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:34.104383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:34.105121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.106270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:34.106270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.106270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:34.203911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:34.204458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:34.204458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:34.205848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.205848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.205848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.205848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.205848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.206312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.206312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:34.208264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:34.208793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:34.209007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.209334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:34.209721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:34.209721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.209721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:34.209721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.209721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:34.209721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:34.209721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.209721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:34.209721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.209721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:34.210275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:34.210885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:34.210885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:34.210885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:34.211161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.211373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.213971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:34.214315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.214558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.214823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:34.215027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:34.215250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:34.215678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.215678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:34.215678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.215678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:34.215678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:34.215678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.215678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:34.215678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.215678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:34.381459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:34.382096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:34.382419 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:34.382704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:34.383237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.383488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.384329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:34.384329 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.384329 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:34.384329 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:34.384329 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.384329 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:34.384329 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.384329 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:34.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:34.387101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.387101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:34.387101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.387101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:34.387101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:34.387101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.387101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:34.387101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.387101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.388370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:34.388984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:34.389299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.389299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:34.389299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.389299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.389981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:34.390446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:34.390940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.390940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:34.390940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.390940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:34.409448 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:34.409848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:34.410096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:34.410188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:34.413815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.414003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:34.414253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.415252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:34.415252 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.415252 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:34.415252 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:34.415252 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.415252 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:34.415252 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.415252 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.417109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.417462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:34.418196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.418196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:34.418632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:34.419053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:34.419426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:34.419426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:34.419426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:39.052848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:39.053259 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:39.053411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:39.054219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:39.111308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.111495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.112763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:39.112763 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.112763 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:39.112763 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:39.112763 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.112763 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:39.112763 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.112763 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.059 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.114005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.114332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:39.114726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.114726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:39.115411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:39.116203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.116784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:39.116784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.116784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:39.205770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:39.207769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:39.207769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:39.211507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.211822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.211973 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.212168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.212747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:39.212974 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.213141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.213260 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.213362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.213463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.215212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:39.215212 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.215212 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:39.215212 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:39.215212 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.215212 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:39.215212 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.215212 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.215759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.216038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:39.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.216373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:39.216431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:39.216732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.217111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:39.217111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.217111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:39.217111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:39.217111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:39.217111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:39.221649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.222140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:39.222478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:39.222725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:39.222991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.223282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.223617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:39.382065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:39.382425 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:39.382425 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:39.382425 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:39.383697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.383697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.384120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:39.384120 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.384120 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:39.384120 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:39.384120 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.384120 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:39.384120 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.384120 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:39.385852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:39.386236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:39.386720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.387038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.387296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:39.387722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:39.388126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.388126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:39.388126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.388126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:39.409890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:39.410210 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:39.410353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:39.410564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:39.411567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.411567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:39.411815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.412832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:39.412832 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.412832 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:39.412832 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:39.412832 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.412832 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:39.412832 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.412832 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.413768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.414279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:39.414715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.414715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:39.414785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:39.415190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:39.418904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:39.418904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:39.418904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:44.054771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:44.055425 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:44.055425 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:44.055959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:44.120475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.120878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.122731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:44.122731 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.122731 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:44.122731 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:44.122731 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.122731 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:44.122731 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.122731 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.068 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.123783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.124554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:44.124884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:44.125173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.125482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:44.125482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:44.125482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.125482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:44.125482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.125482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:44.207866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:44.208871 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:44.208871 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:44.209701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.209836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.210006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.210168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.211093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.211093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.211093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.211093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.211093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.211093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.211593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:44.211593 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.211593 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:44.211593 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:44.211593 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.211593 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:44.211593 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.211593 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.213867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.214110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:44.215693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:44.216390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.216832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:44.216832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:44.216832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.216832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:44.216832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.216832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:44.216832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:44.217397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:44.217583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:44.217760 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:44.218245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.218459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.219618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:44.219618 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.219618 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:44.219618 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:44.219618 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.219618 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:44.219618 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.219618 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:44.220226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:44.220779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.220779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:44.220779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.220779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:44.220779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:44.220779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.220779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:44.220779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.220779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.221201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:44.221652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:44.222046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.222046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:44.222046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.222046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:44.222290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.222858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:44.223354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.223354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:44.223354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:44.223354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.223354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:44.223354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.223354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:44.382510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:44.382859 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:44.382942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:44.383079 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:44.383357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.383568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.383985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:44.383985 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.383985 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:44.383985 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:44.383985 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.383985 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:44.383985 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.383985 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:44.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:44.385816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:44.386292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:44.387197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:44.387686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.388400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:44.410401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:44.411016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:44.411228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:44.411470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:44.414958 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.415233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:44.415509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.416026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:44.416026 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.416026 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:44.416026 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:44.416026 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.416026 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:44.416026 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.416026 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.417830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.418057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:44.418284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:44.418683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:44.419146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:44.419146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:44.419146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:44.419146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:44.419146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:44.419146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:49.055221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:49.055711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:49.055711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:49.056327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:49.088226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.088327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.089059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:49.089059 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.089059 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:49.089059 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:49.089059 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.089059 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:49.089059 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.089059 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.089803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.090131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:49.090448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.090448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:49.090644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:49.090966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.091420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:49.091420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.091420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:49.208411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:49.208815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:49.208927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:49.210630 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.210837 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.211028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.211189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.211347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.211615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.211780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.211892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.212116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.212265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.212328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.213292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:49.213292 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.213292 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:49.213292 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:49.213292 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.213292 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:49.213292 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.213292 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.214535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.214954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:49.215340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.215340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:49.215732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:49.216169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.216776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:49.216776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.216776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:49.217342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:49.217695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:49.217791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:49.217912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:49.218148 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.218286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.218708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:49.218708 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.218708 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:49.218708 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:49.218708 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.218708 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:49.218708 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.218708 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:49.219187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:49.219696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.219696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:49.219696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.219696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:49.219696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:49.219696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.219696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:49.219696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.219696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.219938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:49.220422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:49.220855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.221211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.221751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:49.382962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:49.383282 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:49.383697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:49.383943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:49.384340 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.384756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.387618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:49.388114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.388452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.388799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:49.389117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:49.389473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.389978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:49.389978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.389978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:49.389978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:49.389978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.389978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:49.389978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.389978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:49.410999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:49.411600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:49.411895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:49.412174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:49.413531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.413838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:49.414201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.414872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:49.414872 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.414872 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:49.414872 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:49.414872 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.414872 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:49.414872 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.414872 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.417605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.418054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:49.418446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:49.418885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:49.419364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:49.419364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:49.419364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:49.419364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:49.419364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:49.419364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:54.055733 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:54.056379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:54.056567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:54.057467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:54.067713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=242 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:09:54.106918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.107125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:54.109715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:54.110017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.110368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:54.110368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:54.110368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.110368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:54.110368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.110368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:54.110605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:54.111102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.111436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:54.111436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.111436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:54.208880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:54.209219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:54.209332 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:54.210555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.210805 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.210959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-46e99275-40fc-40f2-bf36-87e245d6f956, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.211121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.211255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.211415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.211569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.211733 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.211879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.211983 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.212096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.212702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:54.212702 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.212702 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:54.212702 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:54.212702 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.212702 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:54.212702 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.212702 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.213518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.213868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:54.214177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.214177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:54.214350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:54.214900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.215357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:54.215357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.215357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:54.217832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:54.218224 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:54.218350 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:54.218453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:54.221654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.222150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:54.222611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:54.222949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:54.223283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.223667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.224096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:54.385682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:54.385682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:54.385682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:54.385682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:54.385682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.385682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.385875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:54.385875 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.385875 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:54.385875 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:54.385875 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.385875 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:54.385875 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.385875 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:54.386755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:54.387080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.387080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:54.387080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.387080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:54.387080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:54.387080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.387080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:54.387080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.387080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.387635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:54.387850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:54.388093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.388093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:54.388093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.388093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.388876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:54.389182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:54.389492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.389492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:54.389492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.389492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:54.411844 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:54.412312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:54.412398 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:54.412608 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:54.413469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.413816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:54.414159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.414826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:54.414826 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.414826 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:54.414826 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:54.414826 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.414826 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:54.414826 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.414826 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.415548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.415928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:54.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:54.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:54.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:54.418405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:54.418405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:54.418405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:59.056603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:59.057487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:59.057588 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:59.058260 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:59.115293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.115631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.059 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:59.118437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:59.119020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:59.119689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:59.119988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:59.120389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.120389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:59.120389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.120389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:59.120389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:59.120389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.120389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:59.120389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.120389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:59.209382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:59.210972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:59.211064 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:59.220113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:59.220113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.220543 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.220543 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.220645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.220798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.220856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.220998 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.221134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.221262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.221384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.221525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.224427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:59.224427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.224427 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:59.224427 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:59.224427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.224427 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:59.224427 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.224427 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.225407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.225749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:59.225983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.225983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:59.226394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:59.226701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.227183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:59.227183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.227183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:59.227183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:59.227183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:59.227183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:09:59.228141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.228417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.230431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:59.230431 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.230431 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:59.230431 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:59.230431 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.230431 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:59.230431 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.230431 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:59.231217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:59.231609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.231609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:59.231609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.231609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:59.231609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:59.231609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.231609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:59.231609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.231609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.232100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:59.232369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:59.232702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.232702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:59.232702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.232702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:59.232793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.233102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:59.233409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.233409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:59.233409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:59.233409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.233409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:59.233409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.233409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:59.383942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:59.386069 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:59.386173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:59.386235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:59.387160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.387160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.388171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:59.388171 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.388171 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:59.388171 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:59.388171 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.388171 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:59.388171 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.388171 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:09:59.388341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:09:59.388717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.388717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:09:59.388717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.388717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:09:59.388717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:09:59.388717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.388717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:09:59.388717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.388717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.389183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:59.389510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:59.389946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.389946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:59.389946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.389946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.392547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:59.393004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:59.393416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.393416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:59.393416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.393416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:09:59.412343 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:09:59.412825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:09:59.413016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:09:59.413232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:09:59.413791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.414104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:09:59.414494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.415179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:09:59.415179 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.415179 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:09:59.415179 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:09:59.415179 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.415179 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:09:59.415179 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.415179 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.416220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.416590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:09:59.416950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.416950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:09:59.417321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:09:59.417689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:09:59.417986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:09:59.417986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:09:59.417986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:04.057561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:04.058070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:04.058187 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:04.058837 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:04.343533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:04.343679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:04.384984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.386050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:04.386050 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.386050 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:04.386050 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:04.386050 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.386050 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:04.386050 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.386050 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:04.386235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.328 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.387508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:04.388019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:04.388386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.388853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:04.388853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:04.388853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.388853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:04.388853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.388853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:04.388853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:04.388853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:04.389711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.389711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.389711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.389945 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.389999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.390161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.390295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.390433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.390586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.390733 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.390864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.391008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.391194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:04.392667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:04.393073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:04.393413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:04.394311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.394611 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:04.396869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:04.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.397657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.398013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:04.398298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.398783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:04.399207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.399207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:04.399207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:04.399207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.399207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:04.399207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.399207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:04.399207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:04.399207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:04.399207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:04.399475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.400147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:04.400559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:04.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:04.402074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:04.412805 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:04.413143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:04.413240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:04.413376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:04.413837 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.414130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:04.414428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.415007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:04.415007 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.415007 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:04.415007 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:04.415007 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.415007 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:04.415007 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.415007 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.416083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.416551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:04.416879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.416879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:04.417212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:04.417612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:04.418347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:04.418347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:04.418347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:09.059221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:09.059221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:09.059221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:09.059834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:09.115878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.115878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.116342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:09.116342 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.116342 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:09.116342 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:09.116342 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.116342 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:09.116342 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.116342 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.117006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.117366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:09.117726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.117726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:09.118016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:09.119090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.120271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:09.120271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.120271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:09.388202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:09.390018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:09.390018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:09.390302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.390503 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.390611 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.390759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.390861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.390987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.391084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.391184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.391283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.391378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.391507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.391605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.391726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:09.393363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:09.393944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.394373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:09.394373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:09.394373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.394373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:09.394373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.394373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:09.394874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:09.396826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:09.397372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:09.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:09.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:09.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:09.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:09.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:09.397947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.398306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.401639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:09.402210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.402723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.403164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:09.403539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:09.403988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:09.404595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.404595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:09.404595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.404595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:09.404595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:09.404595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.404595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:09.404595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.404595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:09.404595 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:09.405370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:09.405370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:09.405696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.405885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.409226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:09.409970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.410911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:09.411238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:09.411738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.412315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:09.412315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.412315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:09.412315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:09.412315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.412315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:09.412315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.412315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:09.413194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:09.413542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:09.413775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:09.414274 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:09.414962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.415137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:09.415388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:09.417963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:09.418422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:09.418888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:09.419273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:09.419840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:09.419840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:09.419840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:09.419840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:09.419840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:09.419840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:09.419840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:09.419840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:09.419840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:09.419840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:14.059209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:14.059719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:14.059858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:14.060881 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:14.133984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.134434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.135485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:14.135485 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.135485 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:14.135485 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:14.135485 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.135485 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:14.135485 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.135485 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.076 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.138026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.138296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:14.138520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.138520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:14.139119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:14.140015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.140436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:14.140436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.140436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:14.388878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:14.390265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:14.390265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:14.391289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.391874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.391966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.393808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.393808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.393808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.393808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.393808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.394001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.394001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.394001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.394001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.394272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.394400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.395648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:14.395648 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.395648 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:14.395648 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:14.395648 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.395648 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:14.395648 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.395648 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.396777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.397075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:14.397424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.397424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:14.397961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:14.400163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:14.400539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.401044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:14.401044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.401044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:14.401044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:14.401044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:14.401044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:14.401044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.401044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.401545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:14.401545 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.401545 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:14.401545 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:14.401545 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.401545 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:14.401545 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.401545 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:14.403145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:14.403728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.404122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.404445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:14.404753 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:14.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:14.406277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.406277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:14.406277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:14.406277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.406277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:14.406277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.406277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:14.406277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:14.406277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:14.406277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:14.406606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.407078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.407942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:14.407942 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.407942 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:14.407942 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:14.407942 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.407942 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:14.407942 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.407942 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:14.408913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:14.409304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.409304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:14.409304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.409304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:14.409304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:14.409304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.409304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:14.409304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.409304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.409878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:14.410214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:14.410559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.410559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:14.410559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.410559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.411889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:14.412884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:14.413527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.413527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:14.413527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.413527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:14.413864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:14.414654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:14.414654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:14.414654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:14.415458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.415458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:14.415884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.416959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:14.416959 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.416959 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:14.416959 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:14.416959 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.416959 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:14.416959 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.416959 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.418667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.419084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:14.419496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:14.419925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:14.420318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:14.420318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:14.420318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:14.420318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:14.420318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:14.420318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:19.062963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:19.066648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:19.066648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:19.066648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:19.139343 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.140970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.079 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:19.146834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:19.147373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:19.147755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:19.148118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:19.148639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.148639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:19.148639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.148639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:19.148639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:19.148639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.148639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:19.148639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.148639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:19.389489 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:19.393634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:19.393634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:19.393739 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.394023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.394328 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.394802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.395602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.395602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.395602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.395602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.395602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.395849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.395849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.395849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.396227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.396343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.397113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:19.397113 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.397113 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:19.397113 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:19.397113 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.397113 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:19.397113 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.397113 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.401873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:19.402250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.402250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:19.402250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:19.402447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:19.402890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.404726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:19.404726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.404726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:19.404726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:19.404726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:19.404726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:19.404726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.404726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.405705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:19.405705 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.405705 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:19.405705 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:19.405705 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.405705 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:19.405705 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.405705 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:19.405889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:19.407038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:19.407342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.407645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.407875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:19.408078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.408403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:19.409050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.409050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:19.409050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:19.409050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.409050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:19.409050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.409050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:19.409050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:19.409050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:19.409050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:19.409527 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.409858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.412859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.415605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:19.415939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:19.416163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.416163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:19.416163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.416163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:19.416163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.416884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:19.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:19.418164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.418164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:19.418164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.418164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:19.418164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:19.418164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:19.418164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:19.418570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.420796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:19.420796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:19.422085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:19.422553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:19.422970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:19.424094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:19.428279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:19.428279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:19.428279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:19.428279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:19.428279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:19.428279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:19.428279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:19.428279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:19.428279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:19.428279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:24.065010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:24.065547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:24.065744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:24.066683 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:24.135513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.136147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.140335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:24.140335 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.140335 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:24.140335 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:24.140335 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.140335 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:24.140335 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.140335 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.076 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.142205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.142904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:24.143270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:24.143803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.144949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:24.144949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:24.144949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.144949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:24.144949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.144949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:24.392243 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:24.392615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:24.392845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:24.394467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.394793 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.394997 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.395173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.395323 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.395464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.396655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.396655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.396655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.396655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.396655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.396814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.396814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.396814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.396814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.397179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:24.397179 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.397179 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:24.397179 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:24.397179 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.397179 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:24.397179 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.397179 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.401499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.401849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:24.402090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:24.402371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.402797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:24.402797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:24.402797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.402797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:24.402797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.402797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:24.403003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:24.403269 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:24.403432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:24.403507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:24.403975 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.404157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.404653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:24.404653 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.404653 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:24.404653 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:24.404653 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.404653 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:24.404653 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.404653 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:24.405386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:24.405883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.405883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:24.405883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.405883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:24.405883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:24.405883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.405883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:24.405883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.405883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.406155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:24.406533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:24.406980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.407390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.407974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:24.409456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:24.410207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:24.410294 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:24.410465 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:24.410948 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.411189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.411878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:24.411878 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.411878 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:24.411878 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:24.411878 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.411878 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:24.411878 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.411878 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:24.412666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:24.413102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.413102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:24.413102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.413102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:24.413102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:24.413102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.413102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:24.413102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.413102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.413374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:24.413686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:24.414018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.414018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:24.414018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.414018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.414768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:24.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:24.417694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.417694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:24.417694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.417694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:24.417694 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:24.417794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:24.417931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:24.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:24.418762 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.419028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:24.419428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.422243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:24.422243 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.422243 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:24.422243 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:24.422243 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.422243 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:24.422243 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.422243 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.423364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.423750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:24.424118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.424118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:24.424680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:24.424996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:24.425412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:24.425412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:24.425412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:29.066779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:29.066779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:29.067262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:29.067948 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:29.145353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.145353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.145353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:29.145353 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.145353 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:29.145353 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:29.145353 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.145353 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:29.145353 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.145353 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.078 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.146375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.146863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:29.147168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.147168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:29.147751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:29.148188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.148679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:29.148679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.148679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:29.392762 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:29.395647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:29.395831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:29.397681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.397789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.398008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.398202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.398390 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.398571 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.398777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.398907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.399080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.399233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.399410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.399558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.399743 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.399889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.400035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.402153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:29.402153 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.402153 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:29.402153 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:29.402153 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.402153 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:29.402153 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.402153 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.403307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:29.403952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.403952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:29.405131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:29.405531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.405888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:29.405888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.405888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:29.405888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:29.405888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:29.405978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:29.406076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:29.411298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:29.411670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.411910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.412602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:29.412602 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.412602 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:29.412602 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:29.412602 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.412602 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:29.412602 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.412602 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:29.416090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:29.416523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.416849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.417088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:29.417659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.418063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:29.418531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.418531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:29.418531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:29.418531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.418531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:29.418531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.418531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:29.418531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:29.418531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:29.418531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:29.418906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:29.419436 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.419721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.420572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:29.420572 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.420572 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:29.420572 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:29.420572 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.420572 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:29.420572 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.420572 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:29.421459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:29.421990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.421990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:29.421990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.421990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:29.421990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:29.421990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.421990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:29.421990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.421990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.422664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:29.423118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:29.423534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.423534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:29.423534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.423534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.424673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:29.425245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:29.425836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.425836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:29.425836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.425836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:29.425836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:29.425836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:29.425836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:29.426449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.426847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:29.427183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.427798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:29.427798 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.427798 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:29.427798 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:29.427798 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.427798 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:29.427798 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.427798 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.428820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.429233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:29.429644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.429644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:29.429989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:29.430456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:29.431129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:29.431129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:29.431129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:34.067347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:34.067347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:34.067347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:34.067942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:34.116890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.117470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:34.117470 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.117470 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:34.117470 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:34.117470 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.117470 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:34.117470 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.117470 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.118232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.118603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:34.118890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.118890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:34.119282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:34.120095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.120523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:34.120523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.120523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:34.395711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:34.396354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:34.396565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:34.399468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.399468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.399468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.399468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.399804 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.399804 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.399985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.400092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.400235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.400374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.400523 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.400656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.400763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.400861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.400960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.402166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:34.402166 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.402166 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:34.402166 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:34.402166 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.402166 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:34.402166 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.402166 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.403243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.403686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:34.404045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.404045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:34.404651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:34.405058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.405506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:34.405506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.405506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:34.406266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:34.406366 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:34.406476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:34.407265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:34.407265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.408390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:34.408390 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.408390 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:34.408390 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:34.408390 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.408390 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:34.408390 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.408390 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:34.409649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:34.410030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.410030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:34.410030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.410030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:34.410030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:34.410030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.410030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:34.410030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.410030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.411259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:34.411702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:34.412051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.412426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.412897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:34.417999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:34.418514 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:34.419451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:34.419451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:34.419451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.419781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:34.420387 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.420387 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:34.420387 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:34.420387 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.420387 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:34.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:34.421154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:34.421835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.421835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:34.421835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.421835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:34.421835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:34.421835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.421835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:34.421835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.421835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.422059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:34.422351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:34.422695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.422695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:34.422695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.422695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.423525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:34.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:34.424406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.424406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:34.424406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.424406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:34.425954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:34.426407 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:34.426512 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:34.426818 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:34.427094 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.427288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:34.427568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.428115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:34.428115 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.428115 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:34.428115 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:34.428115 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.428115 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:34.428115 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.428115 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.428837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.429149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:34.429461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.429461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:34.429518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:34.429909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:34.430357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:34.430357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:34.430357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:39.067057 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:39.067722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:39.067722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:39.068241 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:39.108216 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.108444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.109012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:39.109012 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.109012 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:39.109012 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:39.109012 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.109012 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:39.109012 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.109012 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.109820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.110176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:39.110531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.110531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:39.111450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:39.111938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.112494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:39.112494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.112494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:39.396538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:39.396640 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:39.396724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:39.398089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.398912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.398912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.398912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.398912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.398912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.399819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.399819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.399819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.399819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.399819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.399927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.399927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.399927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.399927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.400860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:39.400860 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.400860 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:39.400860 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:39.400860 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.400860 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:39.400860 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.400860 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.401262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.401520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:39.402668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:39.403230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.403657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:39.403657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:39.403657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.403657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:39.403657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.403657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:39.406925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:39.406925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:39.406925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:39.406925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:39.407257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.407432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.408082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:39.408082 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.408082 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:39.408082 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:39.408082 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.408082 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:39.408082 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.408082 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:39.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:39.409326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.409326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:39.409326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.409326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:39.409326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:39.409326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.409326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:39.409326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.409326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.411079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:39.411324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:39.411602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.411925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.412447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:39.418476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:39.418786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:39.418925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:39.419087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:39.419544 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.419829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.420442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:39.420442 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.420442 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:39.420442 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:39.420442 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.420442 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:39.420442 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.420442 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:39.421084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:39.421450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.421450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:39.421450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.421450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:39.421450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:39.421450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.421450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:39.421450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.421450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.421958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:39.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:39.422519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.422519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:39.422519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.422519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.422997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:39.424091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:39.424799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.424799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:39.424799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.424799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:39.426549 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:39.426876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:39.427026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:39.427227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:39.427539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.427745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:39.428033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.428454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:39.428454 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.428454 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:39.428454 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:39.428454 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.428454 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:39.428454 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.428454 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.429093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:39.429809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.429809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:39.430103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:39.430601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:39.431614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:39.431614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:39.431614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:44.068960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:44.068960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:44.068960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:44.068960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:44.118072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.118503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:44.120558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:44.120937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:44.121243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:44.121530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:44.121993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.121993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:44.121993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.121993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:44.121993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:44.121993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.121993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:44.121993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.121993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:44.396880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:44.398880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:44.398880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:44.399240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.399510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.399773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.399983 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.400059 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.400175 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.400361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.400441 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.400759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.400759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.400826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.400947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.401114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.401317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.401481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.403380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:44.403380 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.403380 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:44.403380 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:44.403380 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.403380 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:44.403380 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.403380 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.407306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.408015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:44.408419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:44.409312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.409967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:44.409967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:44.409967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.409967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:44.409967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.409967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:44.409967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:44.410630 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:44.410726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:44.410886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:44.411509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.411769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.414882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:44.414882 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.414882 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:44.414882 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:44.414882 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.414882 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:44.414882 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.414882 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:44.415862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:44.416276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.416276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:44.416276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.416276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:44.416276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:44.416276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.416276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:44.416276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.416276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.416962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:44.417606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:44.418071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.418071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:44.418071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.418071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:44.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.420366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:44.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:44.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:44.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:44.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:44.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:44.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:44.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:44.421077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:44.422987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.423285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.424015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:44.424015 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.424015 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:44.424015 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:44.424015 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.424015 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:44.424015 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.424015 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:44.426404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:44.428938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:44.429449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:44.429798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:44.430260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:44.431449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:44.431449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:44.431449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:44.431449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:44.431947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.431947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:44.432260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.432857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:44.432857 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.432857 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:44.432857 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:44.432857 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.432857 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:44.432857 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.432857 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.433868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.434350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:44.434765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:44.435215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:44.435733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:44.435733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:44.435733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:44.435733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:44.435733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:44.435733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:49.067884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:49.068391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:49.069528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:49.069528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:49.129271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.129582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.130339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:49.130339 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.130339 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:49.130339 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:49.130339 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.130339 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:49.130339 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.130339 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.062 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.131556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.131826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:49.132092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.132092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:49.133054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:49.133353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.133676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:49.133676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.133676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:49.398354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:49.398832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:49.399093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:49.400817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.401059 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.401428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.401428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.401428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.401659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.401659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.401786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.401864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.401973 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.402141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.402208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.402308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.402397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.402533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:49.404979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:49.405769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:49.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.407677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:49.408416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:49.408416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.408416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:49.408416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.408416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:49.408416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:49.408416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.408416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:49.408416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.408416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:49.410497 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:49.410901 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:49.411165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:49.411493 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:49.412070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.412778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.413069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:49.413069 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.413069 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:49.413069 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:49.413069 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.413069 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:49.413069 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.413069 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:49.415876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:49.416259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.416521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.416738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:49.419270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.420470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:49.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:49.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:49.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:49.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:49.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:49.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:49.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:49.422955 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:49.422955 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.422955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:49.426480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.426819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.427144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:49.428586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:49.429209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.429678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:49.429678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.429678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:49.429678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:49.429678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.429678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:49.429678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.429678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:49.430881 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:49.431207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:49.431327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:49.431520 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:49.432096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.432301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:49.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.433145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:49.433145 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.433145 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:49.433145 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:49.433145 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.433145 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:49.433145 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.433145 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.433991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.434677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:49.435003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:49.435452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:49.435902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:49.435902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:49.435902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:49.435902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:49.435902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:49.435902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:54.068338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:54.069161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:54.069161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:54.070016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:54.078453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=239 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:10:54.127318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.131449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.132214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:54.132214 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.132214 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:54.132214 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:54.132214 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.132214 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:54.132214 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.132214 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.134016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.134324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:54.134516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.134516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:54.135529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:54.135850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.136241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:54.136241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.136241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:54.399590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:54.400241 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:54.400410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:54.403877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.403877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.403877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.403877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.403877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.404261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.404261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.404261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.404261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.404261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.405057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:54.405057 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.405057 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:54.405057 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:54.405057 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.405057 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:54.405057 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.405057 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.406152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.406529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:54.407149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.407149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:54.407263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:54.407681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.408315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:54.408315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.408315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:54.411131 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:54.411493 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:54.411728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:54.411891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:54.413215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.413215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.413463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:54.413463 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.413463 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:54.413463 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:54.413463 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.413463 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:54.413463 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.413463 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:54.414397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:54.414841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.414841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:54.414841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.414841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:54.414841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:54.414841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.414841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:54.414841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.414841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.415822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:54.416285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:54.416792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.416792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:54.416792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.416792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:54.417505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.418089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:54.418911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.418911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:54.418911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:54.418911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.418911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:54.418911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.418911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:54.421148 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:54.421595 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:54.421728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:54.421881 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:54.423102 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.423102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.423700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:54.423700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.423700 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:54.423700 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:54.423700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.423700 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:54.423700 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.423700 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:54.424374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:54.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:54.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:54.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:54.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:54.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.425160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:54.425434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:54.425791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.425791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:54.425791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.425791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.425991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:54.426458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:54.427119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.427119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:54.427119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.427119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:54.432151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:54.432151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:54.432151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:54.432151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:54.432911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.434013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:54.436340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.437242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:54.437242 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.437242 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:54.437242 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:54.437242 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.437242 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:54.437242 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.437242 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.438503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.438990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:54.439528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:54.439961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:54.440361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:54.440361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:54.440361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:54.440361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:54.440361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:54.440361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:59.068985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:59.069470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:59.069470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:59.070337 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:59.122009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.122156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.122741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:59.122741 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.122741 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:59.122741 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:59.122741 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.122741 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:59.122741 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.122741 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.124049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.124381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:59.124704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:59.125066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.125502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:59.125502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:59.125502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.125502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:59.125502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.125502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:59.400229 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:59.401155 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:59.401275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:59.403142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.403389 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.403702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.403759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.403856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-7bfb6c0c-c34e-42ad-bbf0-10660b6f5421, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.404004 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-73afe338-a378-4a82-903a-1938f290e9b7, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.404107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.404313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.404412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.404530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.404646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.404748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.404847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.404967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.405713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:59.405713 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.405713 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:59.405713 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:59.405713 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.405713 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:59.405713 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.405713 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.407095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:59.407313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.407313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:59.408122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:59.408405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:59.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.408762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:59.413453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:59.413453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:59.413453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:59.413453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:10:59.413453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.413675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.414139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:59.414139 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.414139 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:59.414139 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:59.414139 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.414139 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:59.414139 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.414139 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:59.415097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:59.415413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.415413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:59.415413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.415413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:59.415413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:59.415413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.415413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:59.415413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.415413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:59.416367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:59.416610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.416610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:59.416610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.416610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:59.416861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.417266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:59.417708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.417708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:59.417708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:59.417708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.417708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:59.417708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.417708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:59.421815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:59.422158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:59.422215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:59.422341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:59.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.423293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.423822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:59.423822 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.423822 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:59.423822 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:59.423822 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.423822 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:59.423822 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.423822 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:10:59.424708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:10:59.425045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.425045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:10:59.425045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.425045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:10:59.425045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:10:59.425045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.425045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:10:59.425045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.425045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.425346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:59.425557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:59.425770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.425770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:59.425770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.425770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.426702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:59.427293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:59.428117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.428117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:59.428117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.428117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:10:59.432022 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:10:59.432330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:10:59.432519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:10:59.432671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:10:59.434580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.434804 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:10:59.435370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.437234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:10:59.437234 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.437234 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:10:59.437234 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:10:59.437234 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.437234 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:10:59.437234 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.437234 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.438278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.438690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:10:59.439061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.439061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:10:59.439804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:10:59.440422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:10:59.440952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:10:59.440952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:10:59.440952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:04.069405 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:04.069832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:04.069958 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:04.070774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:04.198825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.198825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.198825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:04.198825 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.198825 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:04.198825 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:04.198825 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.198825 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:04.198825 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.198825 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.129 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.202744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.203332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:04.204225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:04.204667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.205215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:04.205215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:04.205215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.205215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:04.205215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.205215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:04.401279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:04.401794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:04.402694 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:04.408106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.408106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.408106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.408322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.408533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.408533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.408770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.408828 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.408993 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.409123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.409237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.409340 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.409474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.412114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:04.412114 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.412114 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:04.412114 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:04.412114 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.412114 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:04.412114 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.412114 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.412730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.413071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:04.413684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:04.414168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.414869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:04.414869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:04.414869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.414869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:04.414869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.414869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:04.416330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:04.416659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:04.416808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:04.416940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:04.418302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.418302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.418868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:04.418868 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.418868 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:04.418868 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:04.418868 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.418868 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:04.418868 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.418868 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:04.419577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:04.420576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.421115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.422054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:04.422728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.423707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:04.424320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.424320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:04.424320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:04.424320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.424320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:04.424320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.424320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:04.424320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:04.426736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:04.427063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:04.427641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:04.430989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.431350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:04.432098 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.432098 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:04.432098 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:04.432098 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.432098 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:04.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:04.435047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:04.435565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.435565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:04.435565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.435565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:04.435565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:04.435565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.435565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:04.435565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.435565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:04.435565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.436102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:04.436666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:04.437159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:04.437524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:04.438037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:04.438417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:04.439137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.439414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:04.439815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:04.443525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:04.444345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:04.444751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:04.445103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:04.445832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:04.445832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:04.445832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:04.445832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:04.445832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:04.445832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:04.445832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:04.445832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:04.445832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:04.445832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:09.070521 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:09.071071 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:09.071342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:09.072054 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:09.140369 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.140369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.141156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:09.141156 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.141156 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:09.141156 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:09.141156 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.141156 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:09.141156 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.141156 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.070 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.142142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.142501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:09.142875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.142875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:09.143725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:09.144182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.144732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:09.144732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.144732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:09.401744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:09.403393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:09.403610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:09.405232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.405491 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.405677 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.405870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.406043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.406207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.406353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.406501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.406668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.406827 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.407012 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.407215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.407298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.409453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:09.409453 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.409453 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:09.409453 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:09.409453 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.409453 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:09.409453 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.409453 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.410756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.411097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:09.411441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.411441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:09.412095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:09.412587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.412993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:09.412993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.412993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:09.417246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:09.417894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:09.418061 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:09.418226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:09.419952 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.420143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.420708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:09.420708 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.420708 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:09.420708 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:09.420708 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.420708 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:09.420708 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.420708 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:09.421548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:09.422024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.422024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:09.422024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.422024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:09.422024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:09.422024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.422024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:09.422024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.422024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.422725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:09.423331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:09.423955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.424518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.425173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:09.427953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:09.428379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:09.428534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:09.428723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:09.429192 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.429440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.430072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:09.430072 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.430072 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:09.430072 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:09.430072 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.430072 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:09.430072 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.430072 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:09.431036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:09.431420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.431420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:09.431420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.431420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:09.431420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:09.431420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.431420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:09.431420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.431420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.432020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:09.432293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:09.432614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.432614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:09.432614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.432614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.433707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:09.434185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:09.434639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.434639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:09.434639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.434639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:09.438307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:09.438758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:09.438923 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:09.439153 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:09.442431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.442657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:09.444724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.444724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:09.444724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.444724 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:09.444724 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:09.444724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.444724 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:09.444724 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.444724 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.447916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.448605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:09.448961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.448961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:09.449382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:09.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:09.451745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:09.451745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:09.451745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:14.071099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:14.072244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:14.072428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:14.073377 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:14.154218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.154461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.156482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:14.156482 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.156482 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:14.156482 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:14.156482 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.156482 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:14.156482 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.156482 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.084 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.158585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.158999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:14.159312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:14.159914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.160482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:14.160482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:14.160482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.160482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:14.160482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.160482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:14.404148 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:14.407586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:14.407586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:14.410842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.410842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.410842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.411052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.415759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.418438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.418438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.418438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.418438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.418438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.418770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.418770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.418770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.418770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.419048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:14.420029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:14.420029 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.420029 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:14.420029 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:14.420029 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.420029 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:14.420029 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.420029 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.422487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.422876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:14.423160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:14.423440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.424488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:14.424488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:14.424488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.424488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:14.424488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.424488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:14.424488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:14.424488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:14.424488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:14.427312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.427312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.429528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:14.430129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.430539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.432674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:14.433025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:14.434292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:14.435329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:14.435704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:14.436070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.436525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:14.436525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:14.436525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.436525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:14.436525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.436525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.438371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:14.438922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:14.439313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:14.439792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:14.440432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:14.440938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:14.441249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:14.441327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:14.442373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.442636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:14.443044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.443788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:14.443788 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.443788 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:14.443788 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:14.443788 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.443788 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:14.443788 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.443788 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.445514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.446157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:14.446478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.446478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:14.446604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:14.448174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:14.448734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:14.448734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:14.448734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:19.072270 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:19.075110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:19.075110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:19.075110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:19.140053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.140285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.141435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:19.141435 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.141435 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:19.141435 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:19.141435 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.141435 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:19.141435 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.141435 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.068 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.142185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.142498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:19.142883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.142883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:19.143967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:19.144339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.144660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:19.144660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.144660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:19.406868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:19.408008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:19.409528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:19.410262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.410519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.411935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.412172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.412367 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.412537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.412747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.412928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.414290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.414580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.414690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.415530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.415530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.416291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.416566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:19.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:19.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.422252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:19.422252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:19.422252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.422252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:19.422252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.422252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:19.422563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:19.422926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.423308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:19.423308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.423308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:19.423308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:19.427124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:19.427309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:19.427787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:19.428123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.430976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.431587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:19.431587 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.431587 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:19.431587 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:19.431587 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.431587 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:19.431587 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.431587 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:19.433768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:19.434160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.434160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:19.434160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.434160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:19.434160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:19.434160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.434160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:19.434160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.434160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.434401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:19.434665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:19.434983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.434983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:19.434983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.434983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:19.434983 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:19.436647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.437122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:19.439590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.439590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:19.439590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:19.439590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.439590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:19.439590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.439590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:19.439590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:19.439590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:19.439590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:19.440051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.440051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.440051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:19.440051 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.440051 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:19.440051 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:19.440051 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.440051 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:19.440051 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.440051 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:19.442938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:19.443450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.443450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:19.443450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.443450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:19.443450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:19.443450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.443450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:19.443450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.443450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.444218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:19.444995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:19.445356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.445356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:19.445356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.445356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:19.445356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.447362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:19.447920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:19.448516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.448516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:19.448516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.448516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:19.448516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:19.449044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:19.450275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:19.455897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.455897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:19.455897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.456130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:19.456130 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.456130 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:19.456130 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:19.456130 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.456130 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:19.456130 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.456130 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.457101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.457382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:19.457712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.457712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:19.458050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:19.458390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:19.459199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:19.459199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:19.459199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:24.074545 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:24.074545 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:24.074545 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:24.075077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:24.133040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.133981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.135005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:24.135005 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.135005 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:24.135005 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:24.135005 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.135005 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:24.135005 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.135005 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.136205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.136828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:24.137163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.137163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:24.137864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:24.138280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.138783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:24.138783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.138783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:24.407751 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:24.408207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:24.408317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:24.410103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.410103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.410103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.410103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.410467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.410639 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.410781 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.410896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.411037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.411177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.411283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.411382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.411478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.411572 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.411724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.412323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:24.412323 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.412323 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:24.412323 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:24.412323 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.412323 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:24.412323 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.412323 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.413207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.413422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:24.413653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.413653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:24.414638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:24.414936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.415208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:24.415208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.415208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:24.427935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:24.428972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:24.428972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:24.428972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:24.429364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.429767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.431567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:24.431567 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.431567 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:24.431567 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:24.431567 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.431567 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:24.431567 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.431567 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:24.436323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:24.436942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.437349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.437712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:24.438824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.439270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:24.439735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.439735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:24.439735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:24.439735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.439735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:24.439735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.439735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:24.440200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:24.441161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:24.441161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:24.441161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:24.442659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.442659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.443091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:24.443091 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.443091 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:24.443091 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:24.443091 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.443091 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:24.443091 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.443091 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:24.444164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:24.444708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.444708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:24.444708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.444708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:24.444708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:24.444708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.444708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:24.444708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.444708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.446636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:24.446889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:24.447104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.447104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:24.447104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.447104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.448171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:24.449407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:24.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:24.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:24.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:24.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:24.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:24.450075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:24.450927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.451000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:24.451520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.452074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:24.452074 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.452074 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:24.452074 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:24.452074 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.452074 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:24.452074 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.452074 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.452886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.453252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:24.453690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.453690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:24.453756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:24.454183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:24.454739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:24.454739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:24.454739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:29.074006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:29.074376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:29.074417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:29.075238 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:29.141430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.141713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.142291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:29.142291 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.142291 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:29.142291 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:29.142291 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.142291 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:29.142291 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.142291 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.068 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.143024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.143400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:29.143755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.143755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:29.143889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:29.144305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.144805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:29.144805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.144805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:29.408274 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:29.408726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:29.408852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:29.410082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.410246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.410376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.410478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.410593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.410725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.410821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.410912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.411072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.411187 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.411285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.411378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.411591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.411718 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.411814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.412412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:29.412412 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.412412 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:29.412412 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:29.412412 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.412412 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:29.412412 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.412412 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.413204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.413650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:29.413965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.413965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:29.414241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:29.414712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.415196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:29.415196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.415196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:29.428614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:29.431847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:29.431999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:29.432190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:29.433467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.433734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.434424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:29.434424 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.434424 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:29.434424 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:29.434424 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.434424 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:29.434424 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.434424 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:29.435173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:29.435549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.435549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:29.435549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.435549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:29.435549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:29.435549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.435549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:29.435549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.435549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.437894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:29.438228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:29.438552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.438552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:29.438552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.438552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:29.439262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.439656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:29.439938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.439938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:29.439938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:29.439938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.439938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:29.439938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.439938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:29.440728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:29.441003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:29.441104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:29.441251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:29.441771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.442028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.442607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:29.442607 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.442607 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:29.442607 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:29.442607 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.442607 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:29.442607 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.442607 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:29.443500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:29.443992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.443992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:29.443992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.443992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:29.443992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:29.443992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.443992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:29.443992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.443992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.444793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:29.445166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:29.445594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.445594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:29.445594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.445594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.445760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:29.446392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:29.446961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.446961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:29.446961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.446961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:29.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:29.449341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:29.449542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:29.449819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:29.450290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.450651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:29.451099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.451847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:29.451847 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.451847 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:29.451847 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:29.451847 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.451847 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:29.451847 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.451847 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.453342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.453757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:29.454205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:29.454696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:29.455298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:29.455298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:29.455298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:29.455298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:29.455298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:29.455298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:34.074598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:34.078720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:34.078959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:34.081745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.058 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:34.139539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.140219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.140590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:34.140915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.141344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:34.408723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:34.409378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:34.409512 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:34.411347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.411347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.411347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.411347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.411646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.411646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.411725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.411927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.412028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.412028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.412377 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.412537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.412713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.412865 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.413021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.414745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:34.414745 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.414745 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:34.414745 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:34.414745 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.414745 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:34.414745 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.414745 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.416238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.416925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:34.417148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:34.417511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.418048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:34.418048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:34.418048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.418048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:34.418048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.418048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:34.431924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:34.434286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:34.434419 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:34.434637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:34.435248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.435560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.438453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:34.438453 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.438453 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:34.438453 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:34.438453 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.438453 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:34.438453 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.438453 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:34.439364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:34.439693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.439693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:34.439693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.439693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:34.439693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:34.439693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.439693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:34.439693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.439693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:34.440531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:34.440772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.440772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:34.440772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.440772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:34.441352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.441710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:34.442034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.442034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:34.442034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:34.442034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.442034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:34.442034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.442034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:34.442034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:34.443545 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:34.444675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:34.444733 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:34.446409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.447276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.447455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:34.447455 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.447455 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:34.447455 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:34.447455 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.447455 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:34.447455 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.447455 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:34.448143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:34.448463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.448463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:34.448463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.448463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:34.448463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:34.448463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.448463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:34.448463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.448463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.449074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:34.449342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:34.449602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.449602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:34.449602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.449602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.453734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:34.454090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:34.454395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.454395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:34.454395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.454395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:34.454395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:34.454395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:34.454395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:34.454676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:34.455059 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.455387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:34.457118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.463256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:34.463256 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.463256 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:34.463256 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:34.463256 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.463256 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:34.463256 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.463256 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.464654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.464984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:34.465301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.465301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:34.468567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:34.469030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:34.469399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:34.469399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:34.469399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:39.076527 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:39.079091 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:39.079091 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:39.079091 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:39.130978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.131882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:39.134368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:39.134925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:39.135286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:39.135681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:39.136118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.136118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:39.136118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.136118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:39.136118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:39.136118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.136118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:39.136118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.136118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:39.409513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:39.409940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:39.410169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:39.412438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.412669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.412797 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.413017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.414912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.414912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.414912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.414912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.415188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.415188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.415188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.415847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.415847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.415847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.415847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.416300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:39.416300 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.416300 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:39.416300 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:39.416300 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.416300 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:39.416300 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.416300 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.418418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.418660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:39.418868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.418868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:39.421716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:39.422011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.422293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:39.422293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.422293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:39.432782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:39.433160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.435492 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:39.435776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:39.436044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.436260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.436470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:39.437744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.438048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:39.438306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.438306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:39.438306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:39.438306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.438306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:39.438306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.438306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:39.443423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:39.444414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:39.444547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:39.444678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:39.445054 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.445797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.445923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:39.445923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.445923 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:39.445923 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:39.445923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.445923 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:39.445923 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.445923 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:39.446688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:39.447105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.447105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:39.447105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.447105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:39.447105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:39.447105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.447105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:39.447105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.447105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.447339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:39.447638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:39.447977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.447977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:39.447977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.447977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.448502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:39.449021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:39.449437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.449437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:39.449437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.449437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:39.454401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:39.455104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:39.455104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:39.455104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:39.456212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.457534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:39.457850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.459024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:39.459024 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.459024 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:39.459024 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:39.459024 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.459024 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:39.459024 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.459024 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.460681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.461006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:39.461212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.461212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:39.461704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:39.462004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:39.462378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:39.462378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:39.462378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:44.077572 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:44.081104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:44.081298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:44.083046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:44.131013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.131214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.131821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:44.131821 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.131821 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:44.131821 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:44.131821 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.131821 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:44.131821 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.131821 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.132805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.133059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:44.133266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.133266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:44.133957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:44.134340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.134815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:44.134815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.134815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:44.410379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:44.411669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:44.411669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:44.412429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.412634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.412764 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.412872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.413007 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.413136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.413244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.413468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.413578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.413725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.413856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.413994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.414131 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.414295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.414920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:44.414920 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.414920 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:44.414920 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:44.414920 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.414920 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:44.414920 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.414920 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.415741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.415969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:44.416156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.416156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:44.416539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:44.417126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.417655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:44.417655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.417655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:44.433453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:44.433736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:44.433893 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:44.434033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:44.434582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.434857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.435524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:44.435524 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.435524 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:44.435524 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:44.435524 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.435524 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:44.435524 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.435524 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:44.436369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:44.436801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.436801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:44.436801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.436801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:44.436801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:44.436801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.436801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:44.436801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.436801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.437277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:44.437602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:44.437925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.437925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:44.437925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.437925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:44.438771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.439174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:44.439596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.439596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:44.439596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:44.439596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.439596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:44.439596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.439596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:44.443981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:44.444221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:44.444379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:44.444522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:44.445213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.445660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.446336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:44.446336 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.446336 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:44.446336 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:44.446336 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.446336 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:44.446336 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.446336 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:44.447252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:44.447724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.447724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:44.447724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.447724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:44.447724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:44.447724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.447724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:44.447724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.447724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.448209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:44.448542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:44.448883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.448883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:44.448883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.448883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:44.450427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:44.450885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.450885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:44.450885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.450885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:44.454992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:44.455197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:44.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:44.455555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:44.456072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.456394 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:44.456743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.457255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:44.457255 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.457255 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:44.457255 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:44.457255 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.457255 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:44.457255 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.457255 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.459361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.459840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:44.460049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.460049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:44.460526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:44.460890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:44.461174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:44.461174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:44.461174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:49.081011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:49.081810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:49.081810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:49.082517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:49.144238 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.144445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.145783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:49.145783 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.145783 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:49.145783 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:49.145783 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.145783 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:49.145783 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.145783 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.146238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.146433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:49.146644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.146644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:49.147040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:49.147487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.147968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:49.147968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.147968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:49.411098 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:49.411462 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:49.411564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:49.413424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.413534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.413726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.413845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.413966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.414076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.414201 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.414324 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.414468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.414565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.414691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.414793 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.414892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.414992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.415117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.415848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:49.415848 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.415848 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:49.415848 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:49.415848 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.415848 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:49.415848 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.415848 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.416964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.417298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:49.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.417643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:49.418138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:49.418593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.419108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:49.419108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.419108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:49.434039 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:49.435244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:49.435244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:49.435244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:49.435244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.435244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.435836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:49.435836 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.435836 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:49.435836 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:49.435836 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.435836 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:49.435836 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.435836 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:49.436540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:49.436981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.436981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:49.436981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.436981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:49.436981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:49.436981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.436981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:49.436981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.436981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.437409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:49.437775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:49.438171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.438171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:49.438171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.438171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:49.438462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.438906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:49.439392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.439392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:49.439392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:49.439392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.439392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:49.439392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.439392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:49.444255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:49.444473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:49.444582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:49.444725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:49.445141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.445416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.445916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:49.445916 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.445916 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:49.445916 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:49.445916 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.445916 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:49.445916 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.445916 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:49.446667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:49.447082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.447082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:49.447082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.447082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:49.447082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:49.447082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.447082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:49.447082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.447082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.447569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:49.447923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:49.448335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.448335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:49.448335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.448335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.448977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:49.449495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:49.450021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.450021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:49.450021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.450021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:49.455271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:49.455583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:49.455711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:49.455888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:49.456240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.457524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:49.457524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.457665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:49.457665 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.457665 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:49.457665 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:49.457665 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.457665 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:49.457665 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.457665 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.458296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.458674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:49.459160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.459160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:49.459336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:49.459916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:49.460535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:49.460535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:49.460535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:54.081713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:54.082258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:54.082258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:54.083096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:54.093335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=240 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:11:54.124909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.125061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.125547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:54.125547 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.125547 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:54.125547 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:54.125547 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.125547 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:54.125547 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.125547 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.126491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.126931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:54.127317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.127317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:54.127474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:54.128140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.128719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:54.128719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.128719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:54.411718 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:54.412077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:54.412343 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:54.413434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.413610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.413773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.413907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.414043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.414186 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.414322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.414458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.414583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.414725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.414852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.414977 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.415138 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.415280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.415406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.415984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:54.415984 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.415984 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:54.415984 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:54.415984 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.415984 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:54.415984 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.415984 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.416782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.417070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:54.417355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.417355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:54.417746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:54.418100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.418518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:54.418518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.418518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:54.434441 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:54.434867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:54.435001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:54.435063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:54.435489 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.435707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.436154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:54.436154 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.436154 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:54.436154 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:54.436154 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.436154 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:54.436154 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.436154 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:54.436703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:54.436971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.436971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:54.436971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.436971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:54.436971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:54.436971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.436971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:54.436971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.436971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.437274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:54.437583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:54.437957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.437957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:54.437957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.437957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:54.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.438517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:54.438845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.438845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:54.438845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:54.438845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.438845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:54.438845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.438845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:54.444682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:54.445065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:54.445187 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:54.445329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:54.445711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.445927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.446320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:54.446320 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.446320 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:54.446320 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:54.446320 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.446320 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:54.446320 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.446320 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:54.446844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:54.447092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.447092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:54.447092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.447092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:54.447092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:54.447092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.447092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:54.447092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.447092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.447491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:54.447694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:54.447955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.447955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:54.447955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.447955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.448240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:54.448550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:54.448875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.448875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:54.448875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.448875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:54.455737 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:54.456091 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:54.456202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:54.456351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:54.456594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.456829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:54.457310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.457765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:54.457765 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.457765 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:54.457765 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:54.457765 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.457765 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:54.457765 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.457765 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.458478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.458735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:54.458932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.458932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:54.459383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:54.459932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:54.460343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:54.460343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:54.460343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:59.084878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:59.085500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:59.085500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:59.086087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:59.133766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.134065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.134922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:59.134922 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.134922 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:59.134922 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:59.134922 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.134922 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:59.134922 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.134922 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.136591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.136873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:59.137147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.137147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:59.138285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:59.138631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.139133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:59.139133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.139133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:59.412136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:59.412478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:59.412592 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:59.414578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.414892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.415135 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.415325 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.415537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.415756 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.415918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.416089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.416245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.416400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.416554 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.416722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.416880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.417084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.417232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.417857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:59.417857 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.417857 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:59.417857 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:59.417857 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.417857 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:59.417857 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.417857 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.418677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.419005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:59.419330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.419330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:59.419734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:59.420387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.421155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:59.421155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.421155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:59.434966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:59.435887 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:59.436083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:59.436083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:11:59.436776 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.436911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.439737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:59.439737 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.439737 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:59.439737 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:59.439737 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.439737 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:59.439737 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.439737 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:59.440368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:59.441215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.441459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.441683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:59.442451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.442992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:59.443486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.443486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:59.443486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:59.443486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.443486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:59.443486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.443486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:59.445111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:59.445388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:59.445690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:59.445812 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:59.446175 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.446421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.446991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:59.446991 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.446991 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:59.446991 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:59.446991 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.446991 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:59.446991 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.446991 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:11:59.447895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:11:59.448305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.448305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:11:59.448305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.448305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:11:59.448305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:11:59.448305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.448305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:11:59.448305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.448305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.448868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:59.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:59.449807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.449807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:59.449807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.449807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.451163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:59.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:59.452204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.452204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:59.452204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.452204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:11:59.456225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:11:59.456706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:11:59.456838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:11:59.457061 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:11:59.457395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.457663 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:11:59.458079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.458677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:11:59.458677 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.458677 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:11:59.458677 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:11:59.458677 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.458677 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:11:59.458677 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.458677 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.459617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.459833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:11:59.460068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.460068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:11:59.460755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:11:59.461042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:11:59.461325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:11:59.461325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:11:59.461325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:04.086169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:04.086654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:04.086795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:04.087603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:04.139146 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.139408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.141710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:04.141710 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.141710 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:04.141710 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:04.141710 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.141710 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:04.141710 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.141710 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.142042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.142428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:04.142791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.142791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:04.142855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:04.143263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.143734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:04.143734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.143734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:04.412750 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:04.413087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:04.413307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:04.417336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.417808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.418457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:04.418457 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.418457 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:04.418457 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:04.418457 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.418457 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:04.418457 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.418457 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.419732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.420155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:04.420897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.420897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:04.421041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:04.421526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.422066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:04.422066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.422066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:04.436076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:04.436391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:04.437035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:04.437035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:04.438799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.438799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.439082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:04.439082 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.439082 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:04.439082 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:04.439082 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.439082 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:04.439082 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.439082 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:04.439898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:04.440312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.440312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:04.440312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.440312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:04.440312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:04.440312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.440312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:04.440312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.440312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.440537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:04.440805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:04.441095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.441095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:04.441095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.441095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:04.441359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.441923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:04.442447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.442447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:04.442447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:04.442447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.442447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:04.442447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.442447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:04.445578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:04.446022 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:04.446164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:04.446279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:04.447507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.447507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.447804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:04.447804 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.447804 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:04.447804 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:04.447804 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.447804 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:04.447804 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.447804 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:04.448395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:04.449025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.449025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:04.449025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.449025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:04.449025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:04.449025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.449025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:04.449025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.449025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.449277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:04.449577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:04.450031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:04.450517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.451044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:04.456747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:04.457301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:04.457395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:04.457597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:04.458070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.458431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:04.459066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.459881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:04.459881 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.459881 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:04.459881 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:04.459881 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.459881 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:04.459881 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.459881 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.460990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.461330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:04.461717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.461717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:04.462048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:04.462413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:04.462960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:04.462960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:04.462960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:09.086892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:09.087728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:09.087728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:09.088498 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:09.142648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.142953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.143641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:09.143641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.143641 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:09.143641 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:09.143641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.143641 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:09.143641 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.143641 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.056 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.144835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.145388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:09.145754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.145754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:09.145856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:09.146318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.146825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:09.146825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.146825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:09.413112 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:09.414532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:09.414842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:09.416547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.417076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.417492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.417739 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.418001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.418221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.418436 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.418682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.418974 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.419149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.419388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.419581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.419800 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.420033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.420219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.420908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:09.420908 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.420908 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:09.420908 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:09.420908 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.420908 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:09.420908 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.420908 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.424163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.424565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:09.424954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.424954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:09.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:09.425953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.426520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:09.426520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.426520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:09.437484 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:09.437484 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:09.437484 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:09.437484 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:09.438075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.438564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.441990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:09.441990 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.441990 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:09.441990 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:09.441990 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.441990 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:09.441990 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.441990 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:09.443292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:09.443748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.443748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:09.443748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.443748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:09.443748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:09.443748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.443748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:09.443748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.443748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.444248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:09.444597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:09.445004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.445004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:09.445004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.445004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:09.448022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.448552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:09.449053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.449053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:09.449053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:09.449053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.449053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:09.449053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.449053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:09.449053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:09.449053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:09.449309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:09.449309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:09.449975 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.450208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.451679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:09.451679 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.451679 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:09.451679 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:09.451679 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.451679 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:09.451679 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.451679 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:09.452497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:09.452956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.452956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:09.452956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.452956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:09.452956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:09.452956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.452956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:09.452956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.452956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.453285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:09.453602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:09.454016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.454016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:09.454016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.454016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.454242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:09.454703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:09.455266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.455266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:09.455266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.455266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:09.457118 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:09.458396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:09.458514 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:09.458722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:09.459093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.459301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:09.459612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.462434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:09.462434 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.462434 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:09.462434 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:09.462434 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.462434 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:09.462434 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.462434 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.463258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.463674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:09.464109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.464109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:09.464174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:09.464581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:09.465032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:09.465032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:09.465032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:14.087339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:14.090368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:14.090368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:14.091060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:14.176962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.090 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.180613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.181212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:14.181638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.181638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:14.181638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:14.181638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.181638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:14.181638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.181638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:14.182067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:14.182590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.183505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:14.183505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.183505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:14.414637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:14.415077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:14.415244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:14.421775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.421775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.421775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.421775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.422007 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.422007 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.422081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.422178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.422278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.422379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.422510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.422697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.422838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.422979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.423122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.423827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:14.423827 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.423827 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:14.423827 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:14.423827 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.423827 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:14.423827 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.423827 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.425126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:14.425883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.425883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:14.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:14.431269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.431783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:14.431783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.431783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:14.437287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:14.438143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:14.438143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:14.438143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:14.438815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.439195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.440031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:14.440031 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.440031 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:14.440031 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:14.440031 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.440031 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:14.440031 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.440031 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:14.442155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:14.442643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.442643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:14.442643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.442643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:14.442643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:14.442643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.442643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:14.442643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.442643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.443604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:14.444121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:14.444485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.444485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:14.444485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.444485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:14.445255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.445790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:14.446246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.446246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:14.446246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:14.446246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.446246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:14.446246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.446246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:14.449240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:14.449752 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:14.450091 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:14.450185 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:14.450914 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.451108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.451852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:14.451852 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.451852 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:14.451852 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:14.451852 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.451852 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:14.451852 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.451852 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:14.453718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:14.454210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.454210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:14.454210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.454210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:14.454210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:14.454210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.454210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:14.454210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.454210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.454481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:14.455417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:14.455854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:14.456514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.457519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:14.459413 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:14.459798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:14.459924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:14.460215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:14.461500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.461500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:14.462870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.463637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:14.463637 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.463637 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:14.463637 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:14.463637 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.463637 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:14.463637 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.463637 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.464946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.465549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:14.465992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.465992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:14.466119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:14.466636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:14.467080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:14.467080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:14.467080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:19.088426 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:19.089028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:19.089028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:19.089481 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:19.146410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.146809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.147375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:19.147375 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.147375 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:19.147375 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:19.147375 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.147375 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:19.147375 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.147375 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.059 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.148476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.149106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:19.149499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.149499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:19.149573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:19.150069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.150579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:19.150579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.150579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:19.415466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:19.415466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:19.415720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:19.417727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.417727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.417727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.417727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.417727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.418058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.418058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.418127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.418252 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.418405 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.418555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.418939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.419297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.419508 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.419737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.420722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:19.420722 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.420722 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:19.420722 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:19.420722 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.420722 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:19.420722 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.420722 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.421036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.421394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:19.421738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.421738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:19.422495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:19.424002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.424854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:19.424854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.424854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:19.437886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:19.439021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:19.439238 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:19.439238 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:19.439773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.439996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.440547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:19.440547 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.440547 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:19.440547 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:19.440547 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.440547 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:19.440547 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.440547 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:19.441269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:19.441779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.441779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:19.441779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.441779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:19.441779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:19.441779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.441779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:19.441779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.441779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:19.442470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:19.442845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.442845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:19.442845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.442845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:19.443126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.443745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:19.444312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.444312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:19.444312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:19.444312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.444312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:19.444312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.444312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:19.449857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:19.450221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:19.450537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:19.450823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:19.451395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.451855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.452739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:19.452739 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.452739 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:19.452739 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:19.452739 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.452739 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:19.452739 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.452739 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:19.453435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:19.453942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.453942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:19.453942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.453942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:19.453942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:19.453942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.453942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:19.453942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.453942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.454427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:19.454811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:19.455159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.455159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:19.455159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.455159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.455501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:19.456061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:19.456717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.456717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:19.456717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.456717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:19.459874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:19.460190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:19.460575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:19.460575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:19.461840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.462562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:19.463862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.464783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:19.464783 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.464783 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:19.464783 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:19.464783 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.464783 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:19.464783 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.464783 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.466143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.466556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:19.466907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.466907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:19.467875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:19.468541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:19.469039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:19.469039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:19.469039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:24.088908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:24.089301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:24.089331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:24.090150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:24.140290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.140290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:24.142926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:24.143326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:24.143678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:24.143993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:24.144462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.144462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:24.144462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.144462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:24.144462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:24.144462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.144462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:24.144462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.144462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:24.416027 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:24.416386 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:24.418880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:24.418880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.418880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.418880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.418880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.419141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.419391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.419509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.419666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.419771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.419897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.419969 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.420073 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.420170 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.420271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.420806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:24.420806 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.420806 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:24.420806 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:24.420806 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.420806 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:24.420806 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.420806 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.421901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.422392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:24.422733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.422733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:24.423029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:24.423582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.424241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:24.424241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.424241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:24.439650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:24.439650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:24.439650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:24.439905 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:24.440250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.440512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.441121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:24.441121 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.441121 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:24.441121 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:24.441121 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.441121 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:24.441121 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.441121 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:24.443659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:24.444152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.444576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.444896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:24.445197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.445674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:24.448370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.448370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:24.448370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:24.448370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.448370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:24.448370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.448370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:24.450306 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:24.450823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:24.450933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:24.451067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:24.451591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.451878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.452391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:24.452391 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.452391 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:24.452391 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:24.452391 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.452391 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:24.452391 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.452391 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:24.453275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:24.453773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.453773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:24.453773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.453773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:24.453773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:24.453773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.453773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:24.453773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.453773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.456384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:24.456776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:24.457113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:24.457444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.457932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:24.460938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:24.460938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:24.460938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:24.460938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:24.461248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.461471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:24.461804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.463008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:24.463008 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.463008 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:24.463008 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:24.463008 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.463008 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:24.463008 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.463008 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.463318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.463715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:24.464059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.464059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:24.464258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:24.464893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:24.465417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:24.465417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:24.465417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:29.089684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:29.090690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:29.092704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:29.092704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:29.159552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.160889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.162955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:29.162955 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.162955 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:29.162955 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:29.162955 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.162955 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:29.162955 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.162955 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.072 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.163810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.164075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:29.164616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.164616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:29.164799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:29.165184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.165526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:29.165526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.165526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:29.416586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:29.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:29.418115 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:29.439782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:29.441129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.441308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.441476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.441673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.441863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.442023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.442077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.442274 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.442526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.442526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.442735 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-8f57fd3a-18b6-4af0-8870-3ec75a5e84e6, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.442794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.442935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.443025 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.443571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.446044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:29.446044 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.446044 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:29.446044 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:29.446044 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.446044 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:29.446044 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.446044 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.447009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.447390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:29.447833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.447833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:29.448095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:29.448618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.449194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:29.449194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.449194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:29.449194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:29.449194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:29.449194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:29.449194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.449194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.449752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:29.449752 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.449752 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:29.449752 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:29.449752 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.449752 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:29.449752 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.449752 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:29.450513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:29.451017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.451017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:29.451017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.451017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:29.451017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:29.451017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.451017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:29.451017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.451017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.451547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:29.452081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:29.452710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.452710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:29.452710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.452710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:29.452710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:29.453052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.453511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:29.454081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.454081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:29.454081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:29.454081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.454081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:29.454081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.454081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:29.454081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:29.454081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:29.454081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:29.454380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.454380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.454713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:29.454713 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.454713 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:29.454713 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:29.454713 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.454713 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:29.454713 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.454713 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:29.455366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:29.455868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.455868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:29.455868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.455868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:29.455868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:29.455868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.455868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:29.455868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.455868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.456265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:29.456606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:29.457014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.457014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:29.457014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.457014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.457323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:29.457805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:29.458281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.458281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:29.458281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.458281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:29.460738 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:29.461029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:29.461133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:29.461302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:29.461631 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.461862 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:29.462235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.462716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:29.462716 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.462716 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:29.462716 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:29.462716 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.462716 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:29.462716 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.462716 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.463595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.464024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:29.464553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.464553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:29.464826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:29.465375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:29.465917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:29.465917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:29.465917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:34.090603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:34.091027 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:34.091027 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:34.091611 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:34.156066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.156287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.156808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:34.156808 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.156808 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:34.156808 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:34.156808 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.156808 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:34.156808 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.156808 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.066 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.157525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.157806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:34.158049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.158049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:34.158321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:34.158736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.159061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:34.159061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.159061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:34.420802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:34.420802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:34.420802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:34.421523 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.421910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.424784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.424990 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.425181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.425330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.425485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.425827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:34.426999 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.426999 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:34.426999 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:34.426999 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.426999 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:34.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.427265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.427662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:34.427958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.427958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:34.428565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:34.428868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.429257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:34.429257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.429257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:34.448667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:34.449068 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:34.449164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:34.449269 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:34.449700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.449913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.450486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:34.450486 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.450486 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:34.450486 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:34.450486 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.450486 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:34.450486 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.450486 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:34.451217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:34.451662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.451662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:34.451662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.451662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:34.451662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:34.451662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.451662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:34.451662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.451662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.452073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:34.452463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:34.452870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.452870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:34.452870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.452870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:34.456505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.457003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:34.457978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.457978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:34.457978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:34.457978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.457978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:34.457978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.457978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:34.458498 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:34.458780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:34.458918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:34.459074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:34.459612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.459836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.460899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:34.460899 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.460899 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:34.460899 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:34.460899 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.460899 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:34.460899 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.460899 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:34.461900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:34.462403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.462403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:34.462403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.462403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:34.462403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:34.462403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.462403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:34.462403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.462403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.462830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:34.463320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:34.463774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.463774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:34.463774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.463774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:34.463774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.464177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:34.464736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:34.465247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.465247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:34.465247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.465247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:34.465247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:34.465247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:34.465247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:34.465247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.465506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:34.466098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.466878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:34.466878 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.466878 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:34.466878 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:34.466878 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.466878 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:34.466878 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.466878 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.467714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.467938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:34.468138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.468138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:34.469972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:34.470544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:34.471085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:34.471085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:34.471085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:39.091568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:39.091568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:39.091568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:39.092215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:39.150074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.150354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.150978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:39.150978 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.150978 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:39.150978 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:39.150978 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.150978 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:39.150978 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.150978 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.059 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.151778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.152128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:39.152436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.152436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:39.152900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:39.153342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.153934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:39.153934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.153934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:39.419230 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:39.419570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:39.419698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:39.420792 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.421217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.421217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.421318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.421470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.421579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.421865 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.421865 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.422000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.422133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.422453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a790be83-8895-4d50-8d03-bd7ea31469a8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.422453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-84bf0e33-fffd-4a9a-82cb-037320b899a9, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.422567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.422729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.423772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:39.423772 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.423772 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:39.423772 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:39.423772 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.423772 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:39.423772 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.423772 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.423985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.424442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:39.424764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:39.425134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.425578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:39.425578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:39.425578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.425578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:39.425578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.425578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:39.449182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:39.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:39.449568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:39.449768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:39.450356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.450660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.451275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:39.451275 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.451275 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:39.451275 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:39.451275 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.451275 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:39.451275 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.451275 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:39.452009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:39.452402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.452402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:39.452402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.452402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:39.452402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:39.452402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.452402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:39.452402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.452402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.452568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:39.452829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:39.453068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.453068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:39.453068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.453068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:39.453579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.454028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:39.454372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.454372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:39.454372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:39.454372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.454372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:39.454372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.454372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:39.458867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:39.459087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:39.459206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:39.459346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:39.459859 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.460061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.460662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:39.460662 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.460662 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:39.460662 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:39.460662 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.460662 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:39.460662 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.460662 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:39.461367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:39.461785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.461785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:39.461785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.461785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:39.461785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:39.461785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.461785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:39.461785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.461785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.462835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:39.463136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:39.463483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.463483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:39.463483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.463483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.463694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:39.464239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:39.464807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.464807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:39.464807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.464807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:39.464807 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:39.464807 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:39.464807 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:39.465004 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:39.465696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.466154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:39.466595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.467221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:39.467221 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.467221 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:39.467221 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:39.467221 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.467221 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:39.467221 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.467221 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.467901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.468196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:39.468408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.468408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:39.468904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:39.469512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:39.470040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:39.470040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:39.470040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:44.091745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:44.094409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:44.094409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:44.094409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:44.164688 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.165377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.166715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:44.166715 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.166715 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:44.166715 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:44.166715 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.166715 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:44.166715 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.166715 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.073 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.167834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.168334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:44.168565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:44.172918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.173293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:44.173293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:44.173293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.173293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:44.173293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.173293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:44.419759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:44.420178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:44.420358 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:44.422547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.422547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.422547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.422547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.422711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-6ae667e7-cc53-47b9-ac79-8d9b06161625, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.422868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.423042 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.423226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.423387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.423501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.423661 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.423770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.423876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.424751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:44.424751 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.424751 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:44.424751 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:44.424751 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.424751 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:44.424751 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.424751 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.427920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.428219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:44.428567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.428567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:44.429219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:44.429864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.430363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:44.430363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.430363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:44.449609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:44.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:44.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:44.451747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:44.453787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.454071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.454653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:44.454653 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.454653 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:44.454653 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:44.454653 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.454653 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:44.454653 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.454653 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:44.455510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:44.455831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.455831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:44.455831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.455831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:44.455831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:44.455831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.455831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:44.455831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.455831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.456120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:44.456392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:44.456701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.456701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:44.456701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.456701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:44.456966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.457649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:44.458216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.458216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:44.458216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:44.458216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.458216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:44.458216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.458216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:44.460445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:44.460445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:44.460445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:44.460445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:44.460445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.460445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.463112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:44.463112 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.463112 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:44.463112 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:44.463112 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.463112 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:44.463112 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.463112 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:44.463770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:44.464518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:44.465345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.465779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.467222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:44.468100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:44.468643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.468643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:44.468643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.468643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:44.470811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:44.470811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:44.470811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:44.471574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:44.472584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.472829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:44.473124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.474101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:44.474101 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.474101 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:44.474101 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:44.474101 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.474101 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:44.474101 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.474101 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.474730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.475126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:44.475373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.475373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:44.475470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:44.475780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:44.476150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:44.476150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:44.476150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:49.092924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:49.097737 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:49.097737 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:49.099066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:49.153556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.153556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.154093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:49.154093 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.154093 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:49.154093 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:49.154093 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.154093 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:49.154093 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.154093 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.155244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.155516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:49.155823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.155823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:49.156374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:49.156782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.157194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:49.157194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.157194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:49.420341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:49.420755 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:49.421160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:49.423502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.423502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.423646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.423710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.423872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.423981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.424107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.424413 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.424413 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.424413 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.424549 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.424590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:49.425248 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.425248 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:49.425248 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:49.425248 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.425248 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:49.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.426104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.426641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:49.426978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.426978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:49.427284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:49.427748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.428269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:49.428269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.428269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:49.451384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:49.451877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:49.452019 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:49.452227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:49.452575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.453328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.453429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:49.453429 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.453429 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:49.453429 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:49.453429 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.453429 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:49.453429 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.453429 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:49.454155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:49.454978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.455985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.456818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:49.457179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.457766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:49.458865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.458865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:49.458865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:49.458865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.458865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:49.458865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.458865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:49.460067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:49.460417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:49.460562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:49.460720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:49.461136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.461462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.462992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.463596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.464044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:49.464385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:49.464773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.464773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:49.464773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.464773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.465137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:49.465655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:49.466149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.466149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:49.466149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.466149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:49.470748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:49.471422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:49.471541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:49.471768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:49.472398 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.472770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:49.473270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.473992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:49.473992 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.473992 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:49.473992 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:49.473992 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.473992 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:49.473992 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.473992 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.474877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.475292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:49.475602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.475602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:49.475931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:49.476403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:49.477067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:49.477067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:49.477067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:54.096757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:54.097286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:54.097286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:54.099884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:54.104576 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=239 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:12:54.149391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.149616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.150356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:54.150356 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.150356 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:54.150356 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:54.150356 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.150356 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:54.150356 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.150356 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.151248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.151521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:54.151758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.151758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:54.152135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:54.152585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.152924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:54.152924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.152924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:54.421539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:54.421539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:54.421539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:54.422692 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.422692 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.422888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.423876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.423876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.423876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.423876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.423876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.424053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.424053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.424053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.424179 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.424307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.424436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:54.433003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:54.433428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:54.433752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.434108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:54.434523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:54.434523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.434523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:54.434523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.434523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:54.434523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:54.434523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.434523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:54.434523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.434523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:54.453558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:54.453558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:54.453558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:54.453558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:12:54.453558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.453558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.454199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:54.454199 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.454199 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:54.454199 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:54.454199 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.454199 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:54.454199 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.454199 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:54.454891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:54.455325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.455325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:54.455325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.455325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:54.455325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:54.455325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.455325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:54.455325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.455325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.455657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:54.455975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:54.456347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.456347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:54.456347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.456347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:54.457645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.458076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:54.458397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.458397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:54.458397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:54.458397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.458397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:54.458397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.458397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:54.460597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:54.460875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:54.461050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:54.461214 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:54.461801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.462150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.462795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:54.462795 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.462795 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:54.462795 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:54.462795 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.462795 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:54.462795 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.462795 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:12:54.463467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:12:54.463769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.463769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:12:54.463769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.463769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:12:54.463769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:12:54.463769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.463769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:12:54.463769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.463769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.464363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:54.464618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:54.464898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.464898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:54.464898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.464898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.465473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:54.465827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:54.466303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.466303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:54.466303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.466303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:54.471521 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:54.471809 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:54.471993 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:54.472183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:12:54.473126 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.473360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:54.473720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.474306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:54.474306 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.474306 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:54.474306 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:54.474306 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.474306 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:54.474306 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.474306 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.475062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.475297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:54.475489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.475489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:12:54.475821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:12:54.476118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:54.476559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:12:54.476559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:54.476559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:12:57.815642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:57.816113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:12:57.816113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:12:57.828509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 27 16:12:57.828864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Number of inconsistencies found at create/update: networks=12, security_groups=4, routers=9, router_ports=9 {{(pid=92285) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 27 16:12:57.828864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Number of inconsistencies found at delete: router_ports=13 {{(pid=92285) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 27 16:12:57.829177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1a1659c0-4943-4985-b3f1-75f0c26af59f (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:57.861175 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, resource={'id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'name': 'public', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['156facd3-ecc0-4e78-8b24-b1c2e5631a78', '3c538124-e9fa-4f89-905c-f13de628c91d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-27T15:57:46Z', 'updated_at': '2026-05-27T15:57:53Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:57.861354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:57.861917 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1d74a028-7065-4269-8c77-c99e0ea79147, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:57.861917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.862399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:57.862399 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:57.862399 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:57.862399 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:57.862399 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:57.862399 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:57.862399 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.862399 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 1a1659c0-4943-4985-b3f1-75f0c26af59f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:57.863447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:57.863924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:57.864267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:57.864267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.864267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:57.864267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 28588915-92a2-4ece-bd6f-c59cf1c236ce (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:57.899088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28588915-92a2-4ece-bd6f-c59cf1c236ce, resource={'id': '28588915-92a2-4ece-bd6f-c59cf1c236ce', 'name': 'shared', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['73ef14d1-56af-457d-a4cc-cd982d0f7719'], 'standard_attr_id': 26, 'shared': True, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T15:58:53Z', 'updated_at': '2026-05-27T15:58:54Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 65356}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:57.899549 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28588915-92a2-4ece-bd6f-c59cf1c236ce, col_values=(('external_ids', {'neutron:network_name': 'shared', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:57.899667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-854ee673-4c8a-4fe6-bb8f-35261288982d, external_ids_update=None, columns={'tag': 65356}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:57.900024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.900488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:57.900488 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:57.900488 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:57.900488 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:57.900488 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:57.900488 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:57.900488 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.900488 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 28588915-92a2-4ece-bd6f-c59cf1c236ce (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:57.902565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:57.903186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:57.903723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:57.903723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.903723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:57.903723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource dc6c799a-6a63-4a52-bbad-a57c955d28f1 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:57.937780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-dc6c799a-6a63-4a52-bbad-a57c955d28f1, resource={'id': 'dc6c799a-6a63-4a52-bbad-a57c955d28f1', 'name': 'tempest-TestEncryptedCinderVolumes-1935686980-network', 'tenant_id': 'a49e1a1a5ae7428ab31444b5236cdd84', 'project_id': 'a49e1a1a5ae7428ab31444b5236cdd84', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['01a05f80-a753-4d54-ae2c-fa68b1e56609'], 'standard_attr_id': 1044, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:09:24Z', 'updated_at': '2026-05-27T16:09:25Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 41666}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:57.937780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-dc6c799a-6a63-4a52-bbad-a57c955d28f1, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestEncryptedCinderVolumes-1935686980-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:57.938401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-9e419501-f578-4b87-b0ba-f2958e3f0128, external_ids_update=None, columns={'tag': 41666}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:57.938401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.938401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:57.938401 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:57.938401 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:57.938401 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:57.938401 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:57.938401 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:57.938401 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.938401 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource dc6c799a-6a63-4a52-bbad-a57c955d28f1 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:57.939116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:57.939474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:57.940428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:57.940428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:57.940428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:57.940428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e2510e0f-28d1-484d-8605-6e6fbc1e01e2 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:57.999823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e2510e0f-28d1-484d-8605-6e6fbc1e01e2, resource={'id': 'e2510e0f-28d1-484d-8605-6e6fbc1e01e2', 'name': 'private', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2524c773-2910-41f1-b8dd-03d9ac809fd5', '71e0c14f-375e-43c9-bf49-7597bd311ee9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T15:57:42Z', 'updated_at': '2026-05-27T15:57:44Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 8519}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.000090 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e2510e0f-28d1-484d-8605-6e6fbc1e01e2, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.000730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-96ddfe89-0ab3-4b19-b3ee-13a966e7d5c2, external_ids_update=None, columns={'tag': 8519}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.000730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.001080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.001080 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.001080 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.001080 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.001080 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.001080 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.001080 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.001080 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource e2510e0f-28d1-484d-8605-6e6fbc1e01e2 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.002190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.003588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.003935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.003935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.003935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.003935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource b4a825ac-07a8-44f7-b011-8b4ddfae7a17 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.026068 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b4a825ac-07a8-44f7-b011-8b4ddfae7a17, resource={'id': 'b4a825ac-07a8-44f7-b011-8b4ddfae7a17', 'name': 'tempest-TestServerVolumeAttachmentScenario-1372617211-network', 'tenant_id': 'afd4a0289e3e413b8809b31980d38838', 'project_id': 'afd4a0289e3e413b8809b31980d38838', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['24a80260-2d9f-45c7-86fd-590306a1ae70'], 'standard_attr_id': 1338, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:11:15Z', 'updated_at': '2026-05-27T16:11:15Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 44044}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.026413 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b4a825ac-07a8-44f7-b011-8b4ddfae7a17, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestServerVolumeAttachmentScenario-1372617211-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.026616 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-63bb051f-f0e9-4d76-8811-823d2a1bad3f, external_ids_update=None, columns={'tag': 44044}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.026738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.027297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.027297 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.027297 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.027297 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.027297 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.027297 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.027297 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.027297 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource b4a825ac-07a8-44f7-b011-8b4ddfae7a17 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.028204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.028496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.028858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.028858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.028858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.028858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 0a8a20f8-405f-4e8d-9e1e-462cca451d96 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.053018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0a8a20f8-405f-4e8d-9e1e-462cca451d96, resource={'id': '0a8a20f8-405f-4e8d-9e1e-462cca451d96', 'name': 'tempest-TestMinimumBasicScenario-1539690753-network', 'tenant_id': '40eca0afa78d41b8a58536328f687c4c', 'project_id': '40eca0afa78d41b8a58536328f687c4c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['5a4d2d85-3a40-4f92-8aa3-15936de1d281'], 'standard_attr_id': 1070, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:09:28Z', 'updated_at': '2026-05-27T16:09:29Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 50423}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.053281 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0a8a20f8-405f-4e8d-9e1e-462cca451d96, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestMinimumBasicScenario-1539690753-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.053584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8151f414-382b-4aac-92f9-7aaeb0604691, external_ids_update=None, columns={'tag': 50423}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.053664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.054246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.054246 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.054246 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.054246 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.054246 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.054246 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.054246 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.054246 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 0a8a20f8-405f-4e8d-9e1e-462cca451d96 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.055120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.055448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.055781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.055781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.055781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.055781 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e80316c0-2c75-4570-ba81-f83c0472a5f2 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.081978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e80316c0-2c75-4570-ba81-f83c0472a5f2, resource={'id': 'e80316c0-2c75-4570-ba81-f83c0472a5f2', 'name': 'tempest-TestServerVolumeAttachmentScenario-1350196525-network', 'tenant_id': '6f84d6fc9cb54e71902adeeb49c37ea4', 'project_id': '6f84d6fc9cb54e71902adeeb49c37ea4', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['9fa6ae7b-d979-4c57-bd3e-8efa941c2bc5'], 'standard_attr_id': 1321, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:11:08Z', 'updated_at': '2026-05-27T16:11:09Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 60516}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.082263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e80316c0-2c75-4570-ba81-f83c0472a5f2, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestServerVolumeAttachmentScenario-1350196525-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.082463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-b84df874-fc6e-4d4d-b25c-a295606e2bd5, external_ids_update=None, columns={'tag': 60516}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.082558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.083201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.083201 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.083201 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.083201 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.083201 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.083201 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.083201 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.083201 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource e80316c0-2c75-4570-ba81-f83c0472a5f2 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.084574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.084950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.085292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.085292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.085292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.085292 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 240ec554-8b3c-4942-a84c-64f519a383e5 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.115113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-240ec554-8b3c-4942-a84c-64f519a383e5, resource={'id': '240ec554-8b3c-4942-a84c-64f519a383e5', 'name': 'tempest-TestEncryptedCinderVolumes-166373726-network', 'tenant_id': '52efa115fc8a4b61bc35acf70a765fec', 'project_id': '52efa115fc8a4b61bc35acf70a765fec', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['704aacf9-ff21-4cc7-86be-71db6593d748'], 'standard_attr_id': 1093, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:09:34Z', 'updated_at': '2026-05-27T16:09:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 21745}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.115113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-240ec554-8b3c-4942-a84c-64f519a383e5, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestEncryptedCinderVolumes-166373726-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.115436 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-48873876-3686-476c-8232-e00c8b2f48d8, external_ids_update=None, columns={'tag': 21745}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.115436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.115999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.115999 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.115999 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.115999 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.115999 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.115999 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.115999 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.115999 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 240ec554-8b3c-4942-a84c-64f519a383e5 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.117076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.117589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.117962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.117962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.117962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.117962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 5816d49f-1ae0-46d7-bfd9-9c3c7962c3d9 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.145088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-5816d49f-1ae0-46d7-bfd9-9c3c7962c3d9, resource={'id': '5816d49f-1ae0-46d7-bfd9-9c3c7962c3d9', 'name': 'tempest-TestMinimumBasicScenario-1920136298-network', 'tenant_id': '4eddf4cfa1284c7ea0c804dabd66d95a', 'project_id': '4eddf4cfa1284c7ea0c804dabd66d95a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['439670e0-66c3-42dc-9a17-40937c9d0c32'], 'standard_attr_id': 1100, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:09:34Z', 'updated_at': '2026-05-27T16:09:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 42487}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.145681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-5816d49f-1ae0-46d7-bfd9-9c3c7962c3d9, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestMinimumBasicScenario-1920136298-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.145681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-947ed24e-01a0-4844-9154-83e089643855, external_ids_update=None, columns={'tag': 42487}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.146496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.147059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.147059 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.147059 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.147059 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.147059 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.147059 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.147059 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.147059 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 5816d49f-1ae0-46d7-bfd9-9c3c7962c3d9 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.147820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.148403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.149384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.149384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.149384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.149384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource d9adeadc-b8b6-458f-bf77-2b7280588f5e (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.172546 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d9adeadc-b8b6-458f-bf77-2b7280588f5e, resource={'id': 'd9adeadc-b8b6-458f-bf77-2b7280588f5e', 'name': 'tempest-AttachVolumeMultiAttachTest-146042705-network', 'tenant_id': 'c3cc201520714585b116dc1ded73757f', 'project_id': 'c3cc201520714585b116dc1ded73757f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['3ac4f469-baad-4c1d-bbe4-886bc99b376c'], 'standard_attr_id': 1202, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:09:58Z', 'updated_at': '2026-05-27T16:09:59Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 51774}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.172929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d9adeadc-b8b6-458f-bf77-2b7280588f5e, col_values=(('external_ids', {'neutron:network_name': 'tempest-AttachVolumeMultiAttachTest-146042705-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.173316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2dfcffbb-4cf0-44b4-8234-29fecc96390e, external_ids_update=None, columns={'tag': 51774}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.173451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.174184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.174184 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.174184 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.174184 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.174184 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.174184 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.174184 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.174184 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource d9adeadc-b8b6-458f-bf77-2b7280588f5e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.175167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.175533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.175943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.175943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.175943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.175943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource a8adaa9e-cf5a-4cc7-8139-7f32f879a15a (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.200109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-a8adaa9e-cf5a-4cc7-8139-7f32f879a15a, resource={'id': 'a8adaa9e-cf5a-4cc7-8139-7f32f879a15a', 'name': 'tempest-TestVolumeBootPatternV346-2099576176-network', 'tenant_id': '3d3a0e2edd224b38a2ee61f7c8799544', 'project_id': '3d3a0e2edd224b38a2ee61f7c8799544', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['63d2725f-af92-472e-b10a-7b52b5bf16be'], 'standard_attr_id': 1114, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:09:36Z', 'updated_at': '2026-05-27T16:09:36Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 64939}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.200476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-a8adaa9e-cf5a-4cc7-8139-7f32f879a15a, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestVolumeBootPatternV346-2099576176-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.200902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3046e9c3-f331-4785-96b7-a65d37fc332a, external_ids_update=None, columns={'tag': 64939}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.201056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.201743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.201743 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.201743 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.201743 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.201743 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.201743 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.201743 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.201743 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource a8adaa9e-cf5a-4cc7-8139-7f32f879a15a (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.202646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.202899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.203201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.203201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.203201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.203201 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource ec4d4154-3f32-4435-ae8a-0a182411a60e (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.227584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-ec4d4154-3f32-4435-ae8a-0a182411a60e, resource={'id': 'ec4d4154-3f32-4435-ae8a-0a182411a60e', 'name': 'tempest-TestVolumeBootPatternV346-1175664702-network', 'tenant_id': '2f6ba4300ce747dca61c29d3d5b8f35a', 'project_id': '2f6ba4300ce747dca61c29d3d5b8f35a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['20db5a09-ab5c-4599-9ae9-5c907cdbb7bf'], 'standard_attr_id': 1146, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:09:45Z', 'updated_at': '2026-05-27T16:09:46Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32798}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.227584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-ec4d4154-3f32-4435-ae8a-0a182411a60e, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestVolumeBootPatternV346-1175664702-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.227885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-741e7566-bbb7-4d5c-a93d-fd35b0a38b4a, external_ids_update=None, columns={'tag': 32798}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.228005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.228528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.228528 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.228528 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.228528 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.228528 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.228528 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.228528 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.228528 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource ec4d4154-3f32-4435-ae8a-0a182411a60e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.229348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.229863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.230282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.230282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.230282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.230282 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource ce17a23f-0c55-40f9-abcf-c22cf134d503 (type: security_groups) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.252034 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Successfully bumped revision number for resource ce17a23f-0c55-40f9-abcf-c22cf134d503 (type: security_groups) to 3
May 27 16:12:58.252155 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 5b8c7ee9-4db5-4428-af8c-68ab67d8c830 (type: security_groups) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.272498 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Successfully bumped revision number for resource 5b8c7ee9-4db5-4428-af8c-68ab67d8c830 (type: security_groups) to 3
May 27 16:12:58.272647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource dbcce34f-bb8a-421d-86a1-8a7d2bb33222 (type: security_groups) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.294352 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Successfully bumped revision number for resource dbcce34f-bb8a-421d-86a1-8a7d2bb33222 (type: security_groups) to 3
May 27 16:12:58.294469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource be3b283c-37ce-429c-8458-a88d94058cbc (type: security_groups) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.315791 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Successfully bumped revision number for resource be3b283c-37ce-429c-8458-a88d94058cbc (type: security_groups) to 3
May 27 16:12:58.315986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1b22a33d-fe65-48de-8053-1f72edb27cad (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.383720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, resource={'id': '1b22a33d-fe65-48de-8053-1f72edb27cad', 'name': 'tempest-TestMinimumBasicScenario-1920136298-router', 'project_id': '4eddf4cfa1284c7ea0c804dabd66d95a', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.126'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::216'}], 'enable_snat': True}, 'gw_port_id': '5600263a-390a-4b19-be3c-a80c416965ff', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.126', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::216', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:09:35Z', 'updated_at': '2026-05-27T16:09:38Z', 'revision_number': 3, 'tenant_id': '4eddf4cfa1284c7ea0c804dabd66d95a'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.383987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, columns={'external_ids': {'neutron:router_name': 'tempest-TestMinimumBasicScenario-1920136298-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.384143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "5600263a-390a-4b19-be3c-a80c416965ff"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.384247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "5600263a-390a-4b19-be3c-a80c416965ff"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.384343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.384962 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.384962 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.384962 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.384962 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.384962 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.384962 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.385261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 1b22a33d-fe65-48de-8053-1f72edb27cad. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 1b22a33d-fe65-48de-8053-1f72edb27cad (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:12:58.386274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.386957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.387603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.387603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.387603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.387603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.387603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.387603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.387603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.387603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.387603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.387603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e68e2d87-4bfd-4c73-960a-b6d3193d2f9d (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.471564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, resource={'id': 'e68e2d87-4bfd-4c73-960a-b6d3193d2f9d', 'name': 'tempest-TestEncryptedCinderVolumes-1935686980-router', 'project_id': 'a49e1a1a5ae7428ab31444b5236cdd84', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.169'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::292'}], 'enable_snat': True}, 'gw_port_id': '733d57d2-3926-45a6-8788-f020194dc56c', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.169', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::292', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:09:26Z', 'updated_at': '2026-05-27T16:09:30Z', 'revision_number': 3, 'tenant_id': 'a49e1a1a5ae7428ab31444b5236cdd84'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.471947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, columns={'external_ids': {'neutron:router_name': 'tempest-TestEncryptedCinderVolumes-1935686980-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.471947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "733d57d2-3926-45a6-8788-f020194dc56c"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.471947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "733d57d2-3926-45a6-8788-f020194dc56c"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.471947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.472635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.472635 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.472635 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.472635 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.472635 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.472635 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.472635 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.472635 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.472928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router e68e2d87-4bfd-4c73-960a-b6d3193d2f9d. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource e68e2d87-4bfd-4c73-960a-b6d3193d2f9d (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:12:58.474598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.475169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.475614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.475614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.475614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.475614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.475614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.475614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.475614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.475614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.475614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.475614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource f6a17c24-60ec-42aa-b567-786a451d36cc (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.579278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, resource={'id': 'f6a17c24-60ec-42aa-b567-786a451d36cc', 'name': 'tempest-TestServerVolumeAttachmentScenario-1372617211-router', 'project_id': 'afd4a0289e3e413b8809b31980d38838', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.100'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::3cb'}], 'enable_snat': True}, 'gw_port_id': 'e6dec0ad-e7e9-429d-b499-4aba52ec4762', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.100', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::3cb', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:11:16Z', 'updated_at': '2026-05-27T16:11:19Z', 'revision_number': 3, 'tenant_id': 'afd4a0289e3e413b8809b31980d38838'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.579357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, columns={'external_ids': {'neutron:router_name': 'tempest-TestServerVolumeAttachmentScenario-1372617211-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.579357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "e6dec0ad-e7e9-429d-b499-4aba52ec4762"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.579357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "e6dec0ad-e7e9-429d-b499-4aba52ec4762"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.579357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.579357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.579357 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.579357 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.579357 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.579529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.579529 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.579529 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.579529 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.579529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router f6a17c24-60ec-42aa-b567-786a451d36cc. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource f6a17c24-60ec-42aa-b567-786a451d36cc (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:12:58.580693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.580975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.581280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.581280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.581280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.581280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.581280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.581280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.581280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.581280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.581280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.581280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 4662b378-9a3d-458b-b74e-9433d4bb6780 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.674477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, resource={'id': '4662b378-9a3d-458b-b74e-9433d4bb6780', 'name': 'tempest-TestServerVolumeAttachmentScenario-1350196525-router', 'project_id': '6f84d6fc9cb54e71902adeeb49c37ea4', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.109'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::e9'}], 'enable_snat': True}, 'gw_port_id': '610ecf1c-5a5c-4559-b303-28748c8c094d', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.109', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::e9', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:11:10Z', 'updated_at': '2026-05-27T16:11:12Z', 'revision_number': 3, 'tenant_id': '6f84d6fc9cb54e71902adeeb49c37ea4'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.674675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, columns={'external_ids': {'neutron:router_name': 'tempest-TestServerVolumeAttachmentScenario-1350196525-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.674675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "610ecf1c-5a5c-4559-b303-28748c8c094d"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.674675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "610ecf1c-5a5c-4559-b303-28748c8c094d"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.674675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.674675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.674675 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.674675 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.674675 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.674916 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.674916 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.674916 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.674916 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.674916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 4662b378-9a3d-458b-b74e-9433d4bb6780. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 4662b378-9a3d-458b-b74e-9433d4bb6780 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:12:58.675880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.676242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.676461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.676461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.676461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.676461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.676461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.676461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.676461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.676461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.676461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.676461 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 34aface9-0038-4177-940c-b8574909b4e1 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.761507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-34aface9-0038-4177-940c-b8574909b4e1, resource={'id': '34aface9-0038-4177-940c-b8574909b4e1', 'name': 'tempest-TestMinimumBasicScenario-1539690753-router', 'project_id': '40eca0afa78d41b8a58536328f687c4c', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.146'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::335'}], 'enable_snat': True}, 'gw_port_id': '38733f67-cd59-4bb8-acef-1cffd44d0e32', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.146', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::335', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:09:29Z', 'updated_at': '2026-05-27T16:09:32Z', 'revision_number': 3, 'tenant_id': '40eca0afa78d41b8a58536328f687c4c'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.761665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-34aface9-0038-4177-940c-b8574909b4e1, columns={'external_ids': {'neutron:router_name': 'tempest-TestMinimumBasicScenario-1539690753-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.761665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "38733f67-cd59-4bb8-acef-1cffd44d0e32"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.761665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "38733f67-cd59-4bb8-acef-1cffd44d0e32"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.761665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.761793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.761793 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.761793 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.761793 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.761793 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.761793 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.761793 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.761793 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.762028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 34aface9-0038-4177-940c-b8574909b4e1. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 34aface9-0038-4177-940c-b8574909b4e1 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:12:58.763610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.763891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.764168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.764168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.764168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.764168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.764168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.764168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.764168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.764168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.764168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.764168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 4e4bef0c-c448-4148-956d-816e97f6ebb0 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.845350 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, resource={'id': '4e4bef0c-c448-4148-956d-816e97f6ebb0', 'name': 'tempest-TestVolumeBootPatternV346-2099576176-router', 'project_id': '3d3a0e2edd224b38a2ee61f7c8799544', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.217'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::35a'}], 'enable_snat': True}, 'gw_port_id': '71eed332-cd4a-4cdb-8d2e-b6135a27facc', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.217', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::35a', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:09:38Z', 'updated_at': '2026-05-27T16:09:42Z', 'revision_number': 3, 'tenant_id': '3d3a0e2edd224b38a2ee61f7c8799544'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.845822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, columns={'external_ids': {'neutron:router_name': 'tempest-TestVolumeBootPatternV346-2099576176-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.845822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "71eed332-cd4a-4cdb-8d2e-b6135a27facc"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.845822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "71eed332-cd4a-4cdb-8d2e-b6135a27facc"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.845822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.845822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.845822 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.845822 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.845822 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.846049 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.846049 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.846049 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.846049 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.846049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 4e4bef0c-c448-4148-956d-816e97f6ebb0. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 4e4bef0c-c448-4148-956d-816e97f6ebb0 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:12:58.847566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.847974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.848279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.848279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.848279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.848279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.848279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.848279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.848279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.848279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.848279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.848279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e94dade4-cb4d-4aaa-9f8b-642d81c63730 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:58.930915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, resource={'id': 'e94dade4-cb4d-4aaa-9f8b-642d81c63730', 'name': 'tempest-AttachVolumeMultiAttachTest-146042705-router', 'project_id': 'c3cc201520714585b116dc1ded73757f', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.5'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::278'}], 'enable_snat': True}, 'gw_port_id': 'a739750f-d85d-4117-83ac-17a265e2d010', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.5', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::278', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:10:00Z', 'updated_at': '2026-05-27T16:10:03Z', 'revision_number': 3, 'tenant_id': 'c3cc201520714585b116dc1ded73757f'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.931101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, columns={'external_ids': {'neutron:router_name': 'tempest-AttachVolumeMultiAttachTest-146042705-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.931101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "a739750f-d85d-4117-83ac-17a265e2d010"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.931101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "a739750f-d85d-4117-83ac-17a265e2d010"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:58.931101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.931300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:58.931300 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.931300 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:58.931300 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:58.931300 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.931300 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:58.931300 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.931300 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:58.931300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router e94dade4-cb4d-4aaa-9f8b-642d81c63730. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource e94dade4-cb4d-4aaa-9f8b-642d81c63730 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:12:58.932364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:58.932722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:58.933140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:58.933140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:58.933140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:58.933140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:58.933140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:58.933140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:58.933140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:58.933140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:58.933140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:58.933140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e96bfc13-6b39-40c6-ba02-1794c1c7f669 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:59.017550 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, resource={'id': 'e96bfc13-6b39-40c6-ba02-1794c1c7f669', 'name': 'tempest-TestEncryptedCinderVolumes-166373726-router', 'project_id': '52efa115fc8a4b61bc35acf70a765fec', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.10'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::273'}], 'enable_snat': True}, 'gw_port_id': 'a3f5d312-05e6-4ce9-a561-14f2fde17de9', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.10', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::273', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:09:36Z', 'updated_at': '2026-05-27T16:09:39Z', 'revision_number': 3, 'tenant_id': '52efa115fc8a4b61bc35acf70a765fec'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.018420 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, columns={'external_ids': {'neutron:router_name': 'tempest-TestEncryptedCinderVolumes-166373726-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.018420 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "a3f5d312-05e6-4ce9-a561-14f2fde17de9"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.018420 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "a3f5d312-05e6-4ce9-a561-14f2fde17de9"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.018420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.019404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:59.019404 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.019404 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:59.019404 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:59.019404 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.019404 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:59.019404 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.019404 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:59.019683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router e96bfc13-6b39-40c6-ba02-1794c1c7f669. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource e96bfc13-6b39-40c6-ba02-1794c1c7f669 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:12:59.021583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:59.021845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:59.022081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:59.022081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:59.022081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.022081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:59.022081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:59.022081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.022081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:59.022081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.022081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:59.022081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource d778c0c1-38a9-470a-831e-2e2a2f716d11 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:59.097601 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:59.118006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, resource={'id': 'd778c0c1-38a9-470a-831e-2e2a2f716d11', 'name': 'tempest-TestVolumeBootPatternV346-1175664702-router', 'project_id': '2f6ba4300ce747dca61c29d3d5b8f35a', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.49'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::b7'}], 'enable_snat': True}, 'gw_port_id': '3448491d-a9ef-42fd-ae45-d63ea6439591', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.49', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::b7', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:09:46Z', 'updated_at': '2026-05-27T16:09:48Z', 'revision_number': 3, 'tenant_id': '2f6ba4300ce747dca61c29d3d5b8f35a'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.118786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, columns={'external_ids': {'neutron:router_name': 'tempest-TestVolumeBootPatternV346-1175664702-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.119167 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3448491d-a9ef-42fd-ae45-d63ea6439591"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.119427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3448491d-a9ef-42fd-ae45-d63ea6439591"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.120226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.120490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:59.120490 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.120490 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:59.120490 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:59.120490 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.120490 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:59.120490 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.120490 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:59.120811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router d778c0c1-38a9-470a-831e-2e2a2f716d11. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource d778c0c1-38a9-470a-831e-2e2a2f716d11 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:12:59.122099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:59.122320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:59.122596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:59.122596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:59.122596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.122596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:59.122596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:59.122596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.122596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:59.122596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.122596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:59.122596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 5600263a-390a-4b19-be3c-a80c416965ff (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:59.256107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-5600263a-390a-4b19-be3c-a80c416965ff, resource={'id': '5600263a-390a-4b19-be3c-a80c416965ff', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:89:28:3b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1b22a33d-fe65-48de-8053-1f72edb27cad', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1108, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.126'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::216'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:09:35Z', 'updated_at': '2026-05-27T16:09:38Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.256207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-5600263a-390a-4b19-be3c-a80c416965ff, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-1b22a33d-fe65-48de-8053-1f72edb27cad'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::216/64', '172.24.5.126/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.256207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=5600263a-390a-4b19-be3c-a80c416965ff, lrouter_port=lrp-5600263a-390a-4b19-be3c-a80c416965ff, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.256207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.256374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:59.256374 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.256374 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:59.256374 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:59.256374 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.256374 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:59.256374 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.256374 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 5600263a-390a-4b19-be3c-a80c416965ff (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.257397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:59.257798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.258255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:59.258255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.258255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:59.258255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource a739750f-d85d-4117-83ac-17a265e2d010 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:59.361294 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-a739750f-d85d-4117-83ac-17a265e2d010, resource={'id': 'a739750f-d85d-4117-83ac-17a265e2d010', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:d2:77:67', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e94dade4-cb4d-4aaa-9f8b-642d81c63730', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1221, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.5'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::278'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:10:00Z', 'updated_at': '2026-05-27T16:10:06Z', 'revision_number': 7, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.361355 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-a739750f-d85d-4117-83ac-17a265e2d010, columns={'external_ids': {'neutron:revision_number': '7', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.5/24', '2001:db8::278/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.362558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=a739750f-d85d-4117-83ac-17a265e2d010, lrouter_port=lrp-a739750f-d85d-4117-83ac-17a265e2d010, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.362558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.368909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:59.368909 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.368909 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:59.368909 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:59.368909 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.368909 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:59.368909 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.368909 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource a739750f-d85d-4117-83ac-17a265e2d010 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.369986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:59.370311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.370783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:59.370783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.370783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:59.370783 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 38733f67-cd59-4bb8-acef-1cffd44d0e32 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:59.421264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:59.453218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:59.461452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:59.472280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:12:59.509060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-38733f67-cd59-4bb8-acef-1cffd44d0e32, resource={'id': '38733f67-cd59-4bb8-acef-1cffd44d0e32', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:6d:a0:15', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '34aface9-0038-4177-940c-b8574909b4e1', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1076, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.146'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::335'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:09:30Z', 'updated_at': '2026-05-27T16:09:34Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.509291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-38733f67-cd59-4bb8-acef-1cffd44d0e32, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-34aface9-0038-4177-940c-b8574909b4e1'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.146/24', '2001:db8::335/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.509502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=38733f67-cd59-4bb8-acef-1cffd44d0e32, lrouter_port=lrp-38733f67-cd59-4bb8-acef-1cffd44d0e32, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.509714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.510196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:59.510196 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.510196 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:59.510196 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:59.510196 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.510196 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:59.510196 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.510196 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 38733f67-cd59-4bb8-acef-1cffd44d0e32 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.511308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:59.511595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.511898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:59.511898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.511898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:59.511898 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 3448491d-a9ef-42fd-ae45-d63ea6439591 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:59.607190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3448491d-a9ef-42fd-ae45-d63ea6439591, resource={'id': '3448491d-a9ef-42fd-ae45-d63ea6439591', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:ff:0f:05', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd778c0c1-38a9-470a-831e-2e2a2f716d11', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1156, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.49'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::b7'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:09:46Z', 'updated_at': '2026-05-27T16:09:50Z', 'revision_number': 7, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.607452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3448491d-a9ef-42fd-ae45-d63ea6439591, columns={'external_ids': {'neutron:revision_number': '7', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.49/24', '2001:db8::b7/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.607805 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3448491d-a9ef-42fd-ae45-d63ea6439591, lrouter_port=lrp-3448491d-a9ef-42fd-ae45-d63ea6439591, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.608097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.608641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:59.608641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.608641 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:59.608641 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:59.608641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.608641 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:59.608641 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.608641 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 3448491d-a9ef-42fd-ae45-d63ea6439591 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.609451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:59.609898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.610308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:59.610308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.610308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:59.610308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 610ecf1c-5a5c-4559-b303-28748c8c094d (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:59.730651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-610ecf1c-5a5c-4559-b303-28748c8c094d, resource={'id': '610ecf1c-5a5c-4559-b303-28748c8c094d', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:39:ea:33', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4662b378-9a3d-458b-b74e-9433d4bb6780', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1326, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.109'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::e9'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:11:10Z', 'updated_at': '2026-05-27T16:11:13Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.732645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-610ecf1c-5a5c-4559-b303-28748c8c094d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-4662b378-9a3d-458b-b74e-9433d4bb6780'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.109/24', '2001:db8::e9/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.732645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=610ecf1c-5a5c-4559-b303-28748c8c094d, lrouter_port=lrp-610ecf1c-5a5c-4559-b303-28748c8c094d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.732645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.732645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:59.732645 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.732645 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:59.732645 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:59.732645 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.732645 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 610ecf1c-5a5c-4559-b303-28748c8c094d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.732931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:59.733135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:59.733367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.733367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:59.733367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.733367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:59.733367 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 733d57d2-3926-45a6-8788-f020194dc56c (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:59.834890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-733d57d2-3926-45a6-8788-f020194dc56c, resource={'id': '733d57d2-3926-45a6-8788-f020194dc56c', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:65:43:c5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e68e2d87-4bfd-4c73-960a-b6d3193d2f9d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1059, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.169'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::292'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:09:26Z', 'updated_at': '2026-05-27T16:09:30Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.835175 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-733d57d2-3926-45a6-8788-f020194dc56c, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::292/64', '172.24.5.169/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.835429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=733d57d2-3926-45a6-8788-f020194dc56c, lrouter_port=lrp-733d57d2-3926-45a6-8788-f020194dc56c, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.835614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.836250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:59.836250 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.836250 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:59.836250 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:59.836250 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.836250 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:59.836250 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.836250 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 733d57d2-3926-45a6-8788-f020194dc56c (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.837404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:59.837729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.838014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:59.838014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.838014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:59.838014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource a3f5d312-05e6-4ce9-a561-14f2fde17de9 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:12:59.980349 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-a3f5d312-05e6-4ce9-a561-14f2fde17de9, resource={'id': 'a3f5d312-05e6-4ce9-a561-14f2fde17de9', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:fb:2f:0b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e96bfc13-6b39-40c6-ba02-1794c1c7f669', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1117, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.10'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::273'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:09:36Z', 'updated_at': '2026-05-27T16:09:42Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.980823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-a3f5d312-05e6-4ce9-a561-14f2fde17de9, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.10/24', '2001:db8::273/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.980823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=a3f5d312-05e6-4ce9-a561-14f2fde17de9, lrouter_port=lrp-a3f5d312-05e6-4ce9-a561-14f2fde17de9, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:12:59.981011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.981555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:12:59.981555 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.981555 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:12:59.981555 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:12:59.981555 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.981555 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:12:59.981555 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.981555 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource a3f5d312-05e6-4ce9-a561-14f2fde17de9 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.982553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:12:59.982941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:12:59.983358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:12:59.983358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:12:59.983358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:12:59.983358 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e6dec0ad-e7e9-429d-b499-4aba52ec4762 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:13:00.089290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-e6dec0ad-e7e9-429d-b499-4aba52ec4762, resource={'id': 'e6dec0ad-e7e9-429d-b499-4aba52ec4762', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:e9:01:39', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f6a17c24-60ec-42aa-b567-786a451d36cc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1345, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.100'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::3cb'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:11:16Z', 'updated_at': '2026-05-27T16:11:20Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.089602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-e6dec0ad-e7e9-429d-b499-4aba52ec4762, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-f6a17c24-60ec-42aa-b567-786a451d36cc'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.100/24', '2001:db8::3cb/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.089874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=e6dec0ad-e7e9-429d-b499-4aba52ec4762, lrouter_port=lrp-e6dec0ad-e7e9-429d-b499-4aba52ec4762, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.090080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.090638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:00.090638 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.090638 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:00.090638 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:00.090638 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.090638 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:00.090638 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.090638 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource e6dec0ad-e7e9-429d-b499-4aba52ec4762 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.091447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:00.092024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.092233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:00.092233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.092233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:00.092233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 71eed332-cd4a-4cdb-8d2e-b6135a27facc (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:13:00.193122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-71eed332-cd4a-4cdb-8d2e-b6135a27facc, resource={'id': '71eed332-cd4a-4cdb-8d2e-b6135a27facc', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:cd:36:9b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4e4bef0c-c448-4148-956d-816e97f6ebb0', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1127, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.217'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::35a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:09:38Z', 'updated_at': '2026-05-27T16:09:43Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.193348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-71eed332-cd4a-4cdb-8d2e-b6135a27facc, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::35a/64', '172.24.5.217/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.193609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=71eed332-cd4a-4cdb-8d2e-b6135a27facc, lrouter_port=lrp-71eed332-cd4a-4cdb-8d2e-b6135a27facc, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.193987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.194569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:00.194569 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.194569 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:00.194569 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:00.194569 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.194569 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:00.194569 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.194569 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 71eed332-cd4a-4cdb-8d2e-b6135a27facc (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.195457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:00.195686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.196006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:00.196006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.196006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:00.196006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1ffd2dc1-814a-4ba2-a764-167527c11e45 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.196006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(1ffd2dc1-814a-4ba2-a764-167527c11e45) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.206649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 25939f7c-c397-475b-8c5f-260598cf1554 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.206649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(25939f7c-c397-475b-8c5f-260598cf1554) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.210124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 2810e0f3-fa51-4f74-bd76-9dce73d27114 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.210428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(2810e0f3-fa51-4f74-bd76-9dce73d27114) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.215648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 2bbb8ff3-18a1-4700-b49a-220c6a0881f0 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.215976 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(2bbb8ff3-18a1-4700-b49a-220c6a0881f0) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.222754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 2ff2e559-2e4b-4102-885d-15fa80595a13 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.223168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(2ff2e559-2e4b-4102-885d-15fa80595a13) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.229256 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 3efb74d3-5d71-46dd-9a59-2342a9210416 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.229256 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(3efb74d3-5d71-46dd-9a59-2342a9210416) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.234533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 46f75301-016c-4e25-8179-25e14b5c4bae (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.236114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(46f75301-016c-4e25-8179-25e14b5c4bae) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.241068 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 53361722-d0c2-4f81-8f91-e067c5cdc146 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.242679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(53361722-d0c2-4f81-8f91-e067c5cdc146) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.246687 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 798996f9-8895-4fe9-9750-5c1fc1915d9a (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.246944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(798996f9-8895-4fe9-9750-5c1fc1915d9a) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.258067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 896470d9-55ac-4bdf-9ed9-43acbefcd53f (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.258437 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(896470d9-55ac-4bdf-9ed9-43acbefcd53f) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.266638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 8be93e28-96c2-40a4-99b0-6df8c779bd81 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.266947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(8be93e28-96c2-40a4-99b0-6df8c779bd81) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.272080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 976929cf-ee16-4c52-add4-4668895a378a (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.272318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(976929cf-ee16-4c52-add4-4668895a378a) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.276843 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource b46889c3-71a3-40bd-b6fe-9ed6a5986c24 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:13:00.277170 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(b46889c3-71a3-40bd-b6fe-9ed6a5986c24) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:13:00.285986 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_for_inconsistencies finished in 2.470 seconds
May 27 16:13:00.286180 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:00.286286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:00.287281 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:00.329576 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.329576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.329816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:00.329816 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.329816 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:00.329816 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:00.329816 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.329816 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:00.329816 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.329816 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.330639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.331038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:00.331334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.331334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:00.331473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:00.331927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.332351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:00.332351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.332351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:00.332351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:00.332351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:00.333778 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.333965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.334155 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.334316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.334468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.334615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.334847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.334990 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.335135 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.335270 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.335404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.335899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.336053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.336195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.336328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.337110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:00.337110 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.337110 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:00.337110 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:00.337110 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.337110 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:00.337110 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.337110 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.338284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.338677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:00.339157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.339157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:00.339736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:00.340168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.340496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:00.340496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.340496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:00.340496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:00.340496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:00.340659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:00.341169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.341735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.342427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:00.342427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.342427 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:00.342427 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:00.342427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.342427 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:00.342427 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.342427 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:00.343331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:00.343707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.343707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:00.343707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.343707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:00.343707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:00.343707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.343707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:00.343707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.343707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.344507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:00.344978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:00.345336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.345336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:00.345336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.345336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:00.345635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.346075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:00.346516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.346516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:00.346516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:00.346516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.346516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:00.346516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.346516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:00.346516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:00.346516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:00.347459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:00.347941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.348223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.348755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:00.348755 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.348755 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:00.348755 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:00.348755 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.348755 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:00.348755 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.348755 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:00.349410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:00.349951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.349951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:00.349951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.349951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:00.349951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:00.349951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.349951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:00.349951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.349951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.350339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:00.350657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:00.350994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.350994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:00.350994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.350994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.351403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:00.352131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:00.352553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.352553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:00.352553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.352553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:00.352553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:00.352553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:00.352553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:00.352553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.352906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:00.353290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.353804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:00.353804 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.353804 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:00.353804 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:00.353804 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.353804 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:00.353804 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.353804 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.354761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.355097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:00.355389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.355389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:00.355936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:00.356314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:00.356746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:00.356746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:00.356746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:05.287017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:05.287017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:05.287487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:05.287827 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:05.332172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:05.340805 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:05.347646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:05.349610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.349610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.349839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:05.349839 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.349839 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:05.349839 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:05.349839 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.349839 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:05.349839 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.349839 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.350985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.351376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:05.351828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.351828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:05.352717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:05.353383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.354240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:05.354240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.354240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:05.354240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:05.354240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:05.354240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:05.355796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.355796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.355796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.355796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.355796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.356127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.356465 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.356830 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.357129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.357418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.357789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.358019 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.358645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.358645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.358776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.359816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:05.359816 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.359816 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:05.359816 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:05.359816 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.359816 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:05.359816 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.359816 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.360805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.361191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:05.361548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.361548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:05.361796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:05.363476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.364089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:05.364089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.364089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:05.364089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:05.364089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:05.364089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:05.364089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.364089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.364885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:05.364885 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.364885 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:05.364885 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:05.364885 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.364885 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:05.364885 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.364885 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:05.365638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:05.366029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.366029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:05.366029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.366029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:05.366029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:05.366029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.366029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:05.366029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.366029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.366350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:05.366757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:05.367139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.367139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:05.367139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.367139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:05.367290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.367838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:05.369533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.369533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:05.369533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:05.369533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.369533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:05.369533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.369533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:05.369533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:05.369533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:05.369533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:05.370066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.370066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.370133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:05.370133 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.370133 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:05.370133 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:05.370133 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.370133 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:05.370133 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.370133 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:05.370909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:05.371357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.371357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:05.371357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.371357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:05.371357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:05.371357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.371357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:05.371357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.371357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.371642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:05.371938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:05.372305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.372305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:05.372305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.372305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.372451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:05.373221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:05.374207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.374207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:05.374207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.374207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:05.374207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:05.374207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:05.374207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:05.374207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.374851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:05.374851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.374851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:05.374851 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.374851 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:05.374851 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:05.374851 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.374851 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:05.374851 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.374851 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.376179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.376484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:05.376969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:05.379361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:05.379879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:05.379879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:05.379879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:05.379879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:05.379879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:05.379879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:10.286982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:10.289819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:10.289935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:10.290685 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:10.354131 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:10.363245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:10.367728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:10.374364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.374364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:10.374638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.375225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:10.375225 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.375225 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:10.375225 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:10.375225 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.375225 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:10.375225 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.375225 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.085 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.375853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.376208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:10.376590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.376590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:10.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:10.377147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.377696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:10.377696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.377696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:10.377696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:10.377696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:10.378720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.379016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.379250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.379447 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.379685 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.379984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.380152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.380312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.380590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.380859 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.381025 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.381181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.381373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.381477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.381667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.382503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:10.382503 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.382503 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:10.382503 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:10.382503 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.382503 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:10.382503 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.382503 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.383543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.384227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:10.384616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.384616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:10.384701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:10.385163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.385782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:10.385782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.385782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:10.385782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:10.385782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:10.385782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:10.385782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.386039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.386921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:10.386921 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.386921 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:10.386921 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:10.386921 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.386921 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:10.386921 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.386921 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:10.387797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:10.388252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.388252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:10.388252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.388252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:10.388252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:10.388252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.388252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:10.388252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.388252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.388748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:10.389119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:10.389482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.389482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:10.389482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.389482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:10.389995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.390505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:10.391010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.391010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:10.391010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:10.391010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.391010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:10.391010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.391010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:10.391010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:10.391010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:10.391010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:10.391296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.391531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.392287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:10.392287 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.392287 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:10.392287 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:10.392287 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.392287 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:10.392287 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.392287 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:10.393503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:10.396143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:10.396530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:10.396950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:10.397254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.397762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:10.398300 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:10.398300 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:10.398300 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:10.398300 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.398300 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:10.398300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.398632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:10.398632 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.398632 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:10.398632 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:10.398632 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.398632 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:10.398632 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.398632 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.399734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.400155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:10.400527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.400527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:10.400974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:10.401472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:10.402017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:10.402017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:10.402017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:15.289751 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:15.290231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:15.290231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:15.290755 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:15.330667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.331057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.331328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:15.331657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.332027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:15.377149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:15.379143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:15.379248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:15.380712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.380907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.381061 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.381217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.381363 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.381528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.381696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.381847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.381995 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.382139 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.382285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.382432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.382591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.382915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.383445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:15.383445 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.383445 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:15.383445 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:15.383445 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.383445 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:15.383445 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.383445 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.384336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.384721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:15.384959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.384959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:15.385466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:15.385809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.386297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:15.386297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.386297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:15.386297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:15.386297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:15.386297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:15.386297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:15.386553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.386553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:15.387089 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.387089 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:15.387089 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:15.387089 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.387089 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:15.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.387089 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:15.389504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:15.390388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.390893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.391571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:15.392975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.393939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:15.394668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.394668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:15.394668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:15.394668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.394668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:15.394668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.394668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:15.394668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:15.394668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:15.394668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:15.395354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:15.395354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.395354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.395894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:15.395894 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.395894 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:15.395894 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:15.395894 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.395894 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:15.395894 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.395894 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:15.397053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:15.398208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:15.398786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:15.399805 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.400941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:15.402267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:15.403114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.403114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:15.403114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.403114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:15.403114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:15.403114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:15.403114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:15.403114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.403434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:15.403434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.404528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:15.404528 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.404528 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:15.404528 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:15.404528 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.404528 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:15.404528 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.404528 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.405581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.405972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:15.406759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.406759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:15.408872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:15.409187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:15.409529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:15.409529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:15.409529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:20.290392 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:20.291189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:20.291189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:20.292191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:20.354227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.355455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.356366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:20.356366 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.356366 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:20.356366 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:20.356366 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.356366 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:20.356366 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.356366 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.065 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.357294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.357739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:20.358063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:20.358505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.359055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:20.359055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:20.359055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.359055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:20.359055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.359055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:20.379259 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:20.379939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:20.379997 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:20.384562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f6a17c24-60ec-42aa-b567-786a451d36cc, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4662b378-9a3d-458b-b74e-9433d4bb6780, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.384986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.384986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:20.384986 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.384986 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:20.384986 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:20.385169 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.385169 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:20.385169 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.385169 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.386697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.387166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:20.387580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.387580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:20.387680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:20.388956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:20.389450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.389832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:20.389832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.389832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:20.389832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:20.389832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:20.389832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:20.390467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.390825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.391708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:20.391708 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.391708 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:20.391708 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:20.391708 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.391708 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:20.391708 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.391708 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:20.392146 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:20.393278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:20.393802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.393802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:20.393802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.393802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:20.393802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:20.393802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.393802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:20.393802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.393802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.394503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:20.394982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:20.395309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.395309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:20.395309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.395309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:20.396941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.397517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:20.398095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.398095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:20.398095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:20.398095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.398095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:20.398095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.398095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:20.398095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:20.398356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:20.398567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:20.399866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.399866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.400098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:20.400098 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.400098 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:20.400098 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:20.400098 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.400098 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:20.400098 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.400098 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:20.401525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:20.402015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.402015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:20.402015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.402015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:20.402015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:20.402015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.402015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:20.402015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.402015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:20.402015 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.403141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:20.403480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:20.403857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.403857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:20.403857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.403857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.404371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:20.404698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:20.405036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.405036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:20.405036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.405036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:20.405036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:20.405036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:20.405036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:20.405474 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.405816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:20.406266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.409065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:20.409065 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.409065 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:20.409065 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:20.409065 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.409065 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:20.409065 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.409065 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.410377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.410825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:20.411134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.411134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:20.413494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:20.413926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:20.414220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:20.414220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:20.414220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:25.290963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:25.291363 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:25.291363 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:25.292269 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:25.381711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:25.389422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:25.395344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.395637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.396420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:25.396420 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.396420 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:25.396420 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:25.396420 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.396420 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:25.396420 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.396420 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.105 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.397591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.397998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:25.398336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.398336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:25.398768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:25.400416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.400903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:25.400903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.400903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:25.400903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:25.400903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:25.401877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.402073 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.402524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.402748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.403018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.403205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.403445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.403636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.403869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.404125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.404345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.404680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.404933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:25.405144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.405927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:25.405927 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.405927 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:25.405927 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:25.405927 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.405927 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:25.405927 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.405927 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.407098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.407502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:25.407864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.407864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:25.408280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:25.408742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.409267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:25.409267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.409267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:25.409267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:25.409267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:25.409267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:25.410190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.410398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.410965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:25.410965 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.410965 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:25.410965 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:25.410965 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.410965 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:25.410965 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.410965 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:25.411868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:25.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:25.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:25.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:25.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:25.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.412122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.412748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:25.412980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:25.413198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.413198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:25.413198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.413198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:25.413973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.414283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:25.414561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.414561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:25.414561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:25.414561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.414561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:25.414561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.414561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:25.414561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:25.414561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:25.414561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:25.415072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.415596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.416226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:25.416226 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.416226 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:25.416226 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:25.416226 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.416226 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:25.416226 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.416226 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:25.417005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:25.417285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.417285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:25.417285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.417285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:25.417285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:25.417285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.417285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:25.417285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.417285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.417922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:25.418125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:25.418364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.418364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:25.418364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.418364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.419162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:25.419482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:25.419788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.419788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:25.419788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.419788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:25.419788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:25.419788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:25.419788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:25.420151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.420384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:25.420671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.421176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:25.421176 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.421176 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:25.421176 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:25.421176 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.421176 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:25.421176 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.421176 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.422140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.422390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:25.422715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.422715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:25.423242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:25.423677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:25.424189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:25.424189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:25.424189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:30.291401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:30.293601 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:30.293601 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:30.293601 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:30.329793 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.330012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:30.332912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:30.333408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:30.333843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:30.334220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:30.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:30.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:30.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:30.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:30.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.334818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:30.400511 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:30.400831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:30.401023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:30.402158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.402338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.402478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.402635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.402780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.403235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.403375 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.403510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.403677 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.403836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.403968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:30.406019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:30.406357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:30.406564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.406815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:30.407141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:30.407141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.407141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:30.407141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.407141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:30.407141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:30.407141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.407141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:30.407141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.407141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:30.408586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:30.408802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:30.408951 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:30.409197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:30.409715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.409930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.412432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:30.413341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.413568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.413821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:30.414037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:30.414366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:30.414675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.414675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:30.414675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.414675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:30.414675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:30.414675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.414675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:30.414675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.414675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:30.414675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:30.415346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:30.415346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:30.415346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:30.415570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.415769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.418634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:30.419033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.419430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.419808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:30.420145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:30.420454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.420813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:30.420813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.420813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:30.420813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:30.420813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.420813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:30.420813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.420813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:30.420813 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:30.421729 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:30.421729 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:30.421729 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:30.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.422268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:30.422646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:30.424462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:30.424777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:30.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:30.425362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:30.425691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:30.425691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:30.425691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:30.425691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:30.425691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:30.425691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:30.425691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:30.425691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:30.425691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:30.425691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:35.293251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:35.293251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:35.293251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:35.293251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:35.333316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.333570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.334514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:35.334514 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.334514 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:35.334514 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:35.334514 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.334514 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:35.334514 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.334514 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.335011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.335253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:35.335475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.335475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:35.335722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:35.336008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.336335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:35.336335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.336335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:35.402072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:35.402846 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:35.403036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:35.405311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.405491 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.405612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.405779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.405879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.406129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.406236 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.406364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.406467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.406569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.406684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.406803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.411117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:35.411117 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.411117 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:35.411117 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:35.411117 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.411117 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:35.411117 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.411117 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:35.411894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.413187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:35.413559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.413559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:35.414045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:35.414442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.415008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:35.415008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.415008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:35.415008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:35.415008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:35.415008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:35.415265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:35.415564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.416058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.416274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:35.416274 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.416274 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:35.416274 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:35.416274 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.416274 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:35.416274 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.416274 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:35.417255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:35.417647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.417993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.418191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:35.419851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.420295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:35.420651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.420651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:35.420651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:35.420651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.420651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:35.420651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.420651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:35.420651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:35.420651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:35.420651 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:35.421015 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.421015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.421676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:35.422082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:35.422082 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.422082 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:35.422082 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:35.422082 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.422082 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:35.422082 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.422082 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:35.423301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:35.423693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.423693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:35.423693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.423693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:35.423693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:35.423693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.423693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:35.423693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.423693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.424601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:35.424853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:35.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:35.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.425142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.426576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:35.426950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:35.427374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.427374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:35.427374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.427374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:35.427374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:35.427374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:35.427502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:35.428384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.428646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:35.428973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.429516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:35.429516 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.429516 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:35.429516 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:35.429516 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.429516 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:35.429516 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.429516 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.430445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.430840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:35.431221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.431221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:35.431500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:35.432610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:35.433150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:35.433150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:35.433150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:40.292892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:40.293336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:40.293336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:40.294089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:40.344832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.345072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.345555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:40.345555 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.345555 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:40.345555 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:40.345555 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.345555 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:40.345555 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.345555 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.346260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.346540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:40.346882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.346882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:40.347052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:40.347416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.347802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:40.347802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.347802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:40.402919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:40.403294 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:40.403383 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:40.404705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.405274 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-1b22a33d-fe65-48de-8053-1f72edb27cad, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.405274 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.405418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.405585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.405750 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.405894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.406074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.406175 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.406321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.406488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-34aface9-0038-4177-940c-b8574909b4e1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.407176 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.407176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.407835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:40.407835 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.407835 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:40.407835 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:40.407835 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.407835 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:40.407835 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.407835 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.409157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.409523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:40.409903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.409903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:40.410323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:40.410613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.411051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:40.411051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.411051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:40.414423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:40.414715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:40.414932 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:40.414985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:40.415323 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.415483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.415897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:40.415897 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.415897 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:40.415897 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:40.415897 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.415897 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:40.415897 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.415897 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:40.416401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:40.416661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.416661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:40.416661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.416661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:40.416661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:40.416661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.416661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:40.416661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.416661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.416975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:40.417221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:40.417432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.417432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:40.417432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.417432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:40.417651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:40.418508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.418508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:40.418508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:40.418508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.418508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:40.418508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.418508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:40.419941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:40.420201 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:40.420339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:40.420339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:40.420629 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.420812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.421171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:40.421171 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.421171 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:40.421171 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:40.421171 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.421171 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:40.421171 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.421171 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:40.421644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:40.421970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.421970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:40.421970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.421970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:40.421970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:40.421970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.421970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:40.421970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.421970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.422316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:40.422532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:40.422810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.422810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:40.422810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.422810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.423054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:40.423411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:40.423779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.423779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:40.423779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.423779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:40.427650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:40.427650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:40.427650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:40.427650 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:40.427801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.427945 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:40.428269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.428681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:40.428681 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.428681 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:40.428681 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:40.428681 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.428681 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:40.428681 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.428681 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.429535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.429949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:40.430329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.430329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:40.431648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:40.431994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:40.432345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:40.432345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:40.432345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:45.293193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:45.295018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:45.295167 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:45.295817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:45.347181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.347359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.347928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:45.347928 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.347928 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:45.347928 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:45.347928 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.347928 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:45.347928 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.347928 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.348811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.349190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:45.349558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.349558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:45.350643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:45.351147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.351686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:45.351686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.351686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:45.403740 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:45.404130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:45.404313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:45.405985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.405985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.406132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.406269 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.406380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.406500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.406612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.406776 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.406957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.407131 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-4e4bef0c-c448-4148-956d-816e97f6ebb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.407378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.408321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:45.408321 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.408321 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:45.408321 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:45.408321 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.408321 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:45.408321 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.408321 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.409839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:45.410037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.410037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:45.410629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:45.410926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.411421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:45.411421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.411421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:45.415153 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:45.415531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:45.415696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:45.415850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:45.416266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.416480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.417765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:45.417765 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.417765 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:45.417765 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:45.417765 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.417765 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:45.417765 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.417765 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:45.419219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:45.419611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.419611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:45.419611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.419611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:45.419611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:45.419611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.419611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:45.419611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.419611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:45.420196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:45.420497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.420497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:45.420497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.420497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:45.420971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.421337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:45.421640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.421640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:45.421640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:45.421640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.421640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:45.421640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.421640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:45.422171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:45.422466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:45.422794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:45.423013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:45.423293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.423531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.424222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:45.424222 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.424222 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:45.424222 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:45.424222 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.424222 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:45.424222 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.424222 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:45.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:45.425188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.425188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:45.425188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.425188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:45.425188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:45.425188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.425188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:45.425188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.425188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.426363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:45.426585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:45.426923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.426923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:45.426923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.426923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.427891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:45.428193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:45.428505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.428505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:45.428505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.428505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:45.428505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:45.428662 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:45.428702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:45.428928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:45.429424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.429688 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:45.430060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.431531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:45.431531 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.431531 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:45.431531 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:45.431531 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.431531 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:45.431531 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.431531 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.432765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:45.432994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.432994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:45.433697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:45.434362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:45.434760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:45.434760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:45.434760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:50.295025 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:50.295699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:50.296472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:50.299658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:50.333128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.333304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.333905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:50.333905 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.333905 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:50.333905 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:50.333905 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.333905 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:50.333905 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.333905 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.334700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.335133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:50.335516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.335516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:50.337104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:50.337670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.338232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:50.338232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.338232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:50.404633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:50.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:50.405322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:50.406222 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.406364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.406489 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.406555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.406748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.406891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.406991 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.407088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.407190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-d778c0c1-38a9-470a-831e-2e2a2f716d11, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.407298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.407419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.408048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:50.408048 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.408048 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:50.408048 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:50.408048 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.408048 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:50.408048 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.408048 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.408818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.409268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:50.409685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.409685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:50.409925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:50.410489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.411542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:50.411542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.411542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:50.415730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:50.416302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:50.416428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:50.416550 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:50.416926 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.417130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.417724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:50.417724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.417724 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:50.417724 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:50.417724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.417724 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:50.417724 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.417724 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:50.418716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:50.419247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.419247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:50.419247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.419247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:50.419247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:50.419247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.419247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:50.419247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.419247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.419561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:50.419973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:50.420408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.420893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.421582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:50.423212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:50.423792 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:50.423959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:50.424154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:50.424637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.424901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.425495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:50.425495 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.425495 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:50.425495 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:50.425495 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.425495 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:50.425495 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.425495 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:50.426561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:50.427068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.427068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:50.427068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.427068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:50.427068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:50.427068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.427068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:50.427068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.427068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.427602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:50.428029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:50.428401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.428401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:50.428401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.428401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.428907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:50.429451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:50.430066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.430066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:50.430066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.430066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:50.430066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:50.430066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:50.430066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:50.430066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:50.430349 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.430873 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:50.431090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.431969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:50.431969 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.431969 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:50.431969 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:50.431969 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.431969 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:50.431969 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.431969 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.432817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.433227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:50.433560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:50.434162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:50.434741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:50.434741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:50.434741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:50.434741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:50.434741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:50.434741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:55.295718 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:55.296337 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:55.296337 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:55.297126 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:55.298088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=1531,neutron:UPDATE=4,neutron:DELETE=13 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:13:55.338565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.339030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.339979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:55.339979 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.339979 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:55.339979 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:55.339979 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.339979 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:55.339979 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.339979 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.340569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.340962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:55.341301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:55.341774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.342442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:55.342442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:55.342442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.342442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:55.342442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.342442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:55.405179 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:55.405586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:55.405725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:55.407693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.407693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.407693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.407693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.407693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.407870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.407870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.407870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.407870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.407870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.407870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:55.407870 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.407870 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:55.407870 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:55.408049 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.408049 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:55.408049 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.408049 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.408384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.408642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:55.408940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.408940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:55.409191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:55.409491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.409944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:55.409944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.409944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:55.416345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:55.416599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:55.416712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:55.416860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:13:55.417341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.417614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.418137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:55.418137 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.418137 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:55.418137 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:55.418137 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.418137 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:55.418137 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.418137 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:55.418766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:55.419071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.419071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:55.419071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.419071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:55.419071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:55.419071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.419071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:55.419071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.419071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.419321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:55.419592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:55.419979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.419979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:55.419979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.419979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:55.420104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.420372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:55.420731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.420731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:55.420731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:55.420731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.420731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:55.420731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.420731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:55.423585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:55.424048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:55.424048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:55.424048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:55.425114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.425114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.425114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:55.425114 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.425114 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:55.425114 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:55.425114 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.425114 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:55.425114 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.425114 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:13:55.425750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:13:55.425997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.425997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:13:55.425997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.425997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:13:55.425997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:13:55.425997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.425997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:13:55.425997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.425997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.426338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:55.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:55.426863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.426863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:55.426863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.426863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.427077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:55.427346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:55.427720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.427720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:55.427720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.427720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:13:55.429289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:13:55.429469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:13:55.429593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:13:55.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:13:55.431176 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.431517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:13:55.432209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.432325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:13:55.432325 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.432325 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:13:55.432325 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:13:55.432325 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.432325 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:13:55.432325 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.432325 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.434542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.434879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:13:55.435609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:13:55.436080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:13:55.436774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:13:55.436774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:13:55.436774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:13:55.436774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:13:55.436774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:13:55.436774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:00.296176 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:00.296748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:00.296748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:00.297830 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:00.345110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.345110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.345110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:00.345110 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.345110 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:00.345110 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:00.345110 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.345110 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:00.345110 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.345110 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.345827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.346471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:00.347119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:00.348005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.348309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:00.348309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:00.348309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.348309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:00.348309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.348309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:00.405716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:00.406304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:00.406304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:00.408972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.409655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.409655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.409655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.409655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.409804 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.409875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.410003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.410132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.410259 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.410398 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.410532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.411404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:00.411404 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.411404 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:00.411404 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:00.411404 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.411404 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:00.411404 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.411404 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.412249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.412485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:00.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.412817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:00.413033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:00.413371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.413712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:00.413712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.413712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:00.417424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:00.417909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:00.418003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:00.418092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:00.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.418781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.419555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:00.419555 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.419555 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:00.419555 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:00.419555 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.419555 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:00.419555 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.419555 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:00.420373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:00.420865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.420865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:00.420865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.420865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:00.420865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:00.420865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.420865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:00.420865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.420865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.421045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:00.421272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:00.421517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.421517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:00.421517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.421517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:00.421791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.422680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:00.422977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.422977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:00.422977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:00.422977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.422977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:00.422977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.422977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:00.423962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:00.424393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:00.424741 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:00.424934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:00.425496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.425781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.428875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:00.428875 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.428875 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:00.428875 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:00.428875 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.428875 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:00.428875 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.428875 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:00.429787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:00.430100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.430100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:00.430100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.430100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:00.430100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:00.430100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.430100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:00.430100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.430100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.431604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:00.431993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:00.432305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.432305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:00.432305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.432305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:00.432305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.432969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:00.433374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:00.433716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.433716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:00.433716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.433716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:00.433716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:00.433716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:00.433716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:00.434481 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.434808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:00.435133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.437312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:00.437312 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.437312 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:00.437312 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:00.437312 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.437312 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:00.437312 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.437312 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.438436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.438768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:00.439018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.439018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:00.440208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:00.440757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:00.441080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:00.441080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:00.441080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:05.296704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:05.297848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:05.298271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:05.300248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:05.347304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.348067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.348067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:05.348067 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.348067 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:05.348067 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:05.348067 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.348067 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:05.348067 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.348067 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.348899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.349168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:05.349379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.349379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:05.351101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:05.351381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.351687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:05.351687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.351687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:05.406303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:05.406776 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:05.407055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:05.408306 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.408529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.408704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.408933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.408992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.409149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.409448 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.409504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.409833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.409833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.409997 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.410171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.410450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.410928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:05.410928 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.410928 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:05.410928 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:05.410928 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.410928 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:05.410928 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.410928 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.411756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.412113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:05.412478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.412478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:05.412901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:05.413228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.413503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:05.413503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.413503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:05.417867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:05.418139 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:05.418268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:05.418405 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:05.418752 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.418958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.419473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:05.419473 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.419473 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:05.419473 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:05.419473 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.419473 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:05.419473 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.419473 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:05.420273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:05.420754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.420754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:05.420754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.420754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:05.420754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:05.420754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.420754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:05.420754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.420754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.420953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:05.421275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:05.421613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.421613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:05.421613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.421613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:05.421749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.422048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:05.422324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.422324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:05.422324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:05.422324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.422324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:05.422324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.422324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:05.424272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:05.424578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:05.424721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:05.424869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:05.426646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.426844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.427325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:05.427325 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.427325 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:05.427325 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:05.427325 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.427325 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:05.427325 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.427325 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:05.427948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:05.428292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.428292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:05.428292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.428292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:05.428292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:05.428292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.428292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:05.428292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.428292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.428883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:05.429082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:05.429309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.429309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:05.429309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.429309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.429956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:05.430294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:05.430667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.430667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:05.430667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.430667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:05.433393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:05.433789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:05.433892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:05.434033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:05.434289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.434538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:05.434932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.435555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:05.435555 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.435555 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:05.435555 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:05.435555 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.435555 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:05.435555 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.435555 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.436468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.436723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:05.436941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.436941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:05.437588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:05.437929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:05.438309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:05.438309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:05.438309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:10.297447 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:10.297937 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:10.298108 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:10.299016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:10.347447 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.347882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.348434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:10.348434 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.348434 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:10.348434 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:10.348434 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.348434 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:10.348434 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.348434 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.349579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.349846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:10.350110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.350110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:10.350690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:10.351235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.351773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:10.351773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.351773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:10.406685 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:10.408439 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:10.408439 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:10.408987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.409224 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.409429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.409593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.409774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.409927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.410074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.410292 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.410492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.410693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.410883 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.411087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.411232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.411724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:10.411724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.411724 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:10.411724 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:10.411724 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.411724 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:10.411724 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.411724 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.412455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.412773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:10.413124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.413124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:10.413540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:10.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.414552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:10.414552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.414552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:10.418288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:10.418585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:10.418771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:10.418896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:10.419233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.419407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.419841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:10.419841 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.419841 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:10.419841 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:10.419841 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.419841 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:10.419841 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.419841 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:10.420554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:10.421128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.421128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:10.421128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.421128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:10.421128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:10.421128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.421128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:10.421128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.421128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.421478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:10.421845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:10.422234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.422234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:10.422234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.422234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:10.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.422916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:10.423485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.423485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:10.423485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:10.423485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.423485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:10.423485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.423485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:10.424824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:10.425175 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:10.425303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:10.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:10.426483 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.431517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:10.431517 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.431517 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:10.431517 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:10.431517 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.431517 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:10.431517 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.431517 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:10.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:10.432510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.432510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:10.432510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.432510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:10.432510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:10.432510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.432510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:10.432510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.432510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.432778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:10.433107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:10.433471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:10.433867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:10.434377 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:10.434721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:10.434721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:10.434721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.434721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:10.434986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.435327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:10.435327 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.435327 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:10.435327 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:10.435327 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.435327 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:10.435327 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.435327 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.435961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.436326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:10.436702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:10.437144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:10.437635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:10.437635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:10.437635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:10.437635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:10.437635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:10.437635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:15.297925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:15.298348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:15.298943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:15.299016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:15.334469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.334686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.335490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:15.335490 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.335490 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:15.335490 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:15.335490 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.335490 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:15.335490 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.335490 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.335861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.336151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:15.336346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.336346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:15.336899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:15.337286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.337632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:15.337632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.337632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:15.407246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:15.407810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:15.409276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:15.410298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.410464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.410595 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.410742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.410862 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.410989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.411101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.411338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.411449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.411560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.411690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.411812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.412707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:15.412707 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.412707 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:15.412707 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:15.412707 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.412707 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:15.412707 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.412707 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.413369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:15.413569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.413569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:15.414272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:15.414672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.415056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:15.415056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.415056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:15.418887 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:15.419254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:15.419357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:15.419544 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:15.420692 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.420692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.420692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:15.420692 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.420692 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:15.420692 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:15.420692 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.420692 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:15.420692 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.420692 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:15.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:15.421773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.422100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.422325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:15.422511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.422786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:15.423191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.423191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:15.423191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:15.423191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.423191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:15.423191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.423191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:15.425295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:15.425558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:15.425678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:15.425786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:15.426095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.426320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.426812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:15.426812 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.426812 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:15.426812 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:15.426812 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.426812 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:15.426812 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.426812 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:15.427552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.427552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:15.427552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.427552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:15.427552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:15.427552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.427552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:15.427552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.427552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.428330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:15.428683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:15.428970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:15.429388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.429964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:15.434214 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:15.434673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:15.434814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:15.435036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:15.435406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.435663 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:15.436053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.436596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:15.436596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.436596 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:15.436596 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:15.436596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.436596 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:15.436596 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.436596 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.437499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.437794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:15.438116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.438116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:15.438429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:15.438704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:15.439041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:15.439041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:15.439041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:20.301880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:20.301880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:20.301880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:20.301880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:20.359336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.359592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.360307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:20.360307 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.360307 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:20.360307 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:20.360307 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.360307 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:20.360307 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.360307 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.361090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.361401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:20.361671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.361671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:20.362029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:20.362409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:20.407801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:20.408249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:20.408362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:20.409739 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.410040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.410192 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.410327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.410446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.410555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.410679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.410787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.410951 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.411071 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.411175 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.411277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.411380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.411867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:20.411867 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.411867 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:20.411867 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:20.411867 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.411867 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:20.411867 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.411867 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.412553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.412883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:20.413105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.413105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:20.413312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:20.413665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.414034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:20.414034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.414034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:20.419683 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:20.419922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:20.420010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:20.420102 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:20.420880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.421056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.421529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:20.421529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.421529 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:20.421529 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:20.421529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.421529 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:20.421529 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.421529 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:20.422258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:20.422558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.422558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:20.422558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.422558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:20.422558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:20.422558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.422558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:20.422558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.422558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.422870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:20.423099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:20.423416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.423416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:20.423416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.423416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:20.423552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.424042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:20.424366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.424366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:20.424366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:20.424366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.424366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:20.424366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.424366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:20.425636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:20.425906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:20.426020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:20.426170 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:20.426528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.426735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.427393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:20.427393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.427393 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:20.427393 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:20.427393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.427393 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:20.427393 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.427393 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:20.427932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:20.428239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.428239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:20.428239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.428239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:20.428239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:20.428239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.428239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:20.428239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.428239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.428734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:20.428996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:20.429216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.429216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:20.429216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.429216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.429457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:20.429802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:20.430144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.430144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:20.430144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.430144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:20.434577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:20.434762 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:20.434858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:20.434978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:20.435431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.435721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:20.436133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.436781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:20.436781 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.436781 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:20.436781 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:20.436781 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.436781 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:20.436781 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.436781 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.437454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.437748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:20.438094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:20.438367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:20.439317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:20.439317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:20.439317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:20.439317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:20.439317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:20.439317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:25.298947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:25.299475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:25.299475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:25.300030 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:25.360345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.360604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.361250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:25.361250 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.361250 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:25.361250 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:25.361250 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.361250 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:25.361250 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.361250 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.062 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.362332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.362681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:25.362898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.362898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:25.363526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:25.364042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.364557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:25.364557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.364557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:25.409021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:25.409021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:25.409021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:25.411980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.412285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.412412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.412564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.412825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.414307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.414307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.414307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.414307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.414307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.414572 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.415438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.416164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:25.416720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:25.417124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.417604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:25.417604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:25.417604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.417604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:25.417604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.417604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:25.420048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:25.420303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:25.420423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:25.420675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:25.422052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.422208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.422779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:25.422779 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.422779 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:25.422779 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:25.422779 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.422779 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:25.422779 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.422779 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:25.423785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:25.424744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.425261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.425599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:25.426000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.426448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:25.426998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.426998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:25.426998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:25.426998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.426998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:25.426998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.426998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:25.426998 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:25.426998 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:25.426998 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:25.427404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:25.427404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.427480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.428028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:25.428028 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.428028 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:25.428028 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:25.428028 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.428028 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:25.428028 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.428028 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:25.428799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:25.429224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.429224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:25.429224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.429224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:25.429224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:25.429224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.429224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:25.429224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.429224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.429746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:25.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:25.430486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.430486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:25.430486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.430486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.430863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:25.431334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:25.431848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.431848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:25.431848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.431848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:25.434909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:25.435931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:25.435931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:25.437023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:25.437547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.437791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:25.438060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.438551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:25.438551 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.438551 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:25.438551 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:25.438551 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.438551 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:25.438551 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.438551 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.439530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.440414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:25.440924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:25.441316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:25.441816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:25.441816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:25.441816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:25.441816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:25.441816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:25.441816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:30.299380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:30.299749 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:30.299814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:30.300658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:30.350174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.350404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.351064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:30.351064 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.351064 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:30.351064 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:30.351064 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.351064 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:30.351064 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.351064 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.352044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.352288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:30.352497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.352497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:30.353025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:30.353284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.353573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:30.353573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.353573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:30.408798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:30.409141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:30.409287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:30.411182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.411310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.411471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.411589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.411742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.411850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.412452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.412452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.412452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.412452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.412452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.412612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.412747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.413348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:30.413348 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.413348 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:30.413348 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:30.413348 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.413348 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:30.413348 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.413348 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.414264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.414500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:30.414744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.414744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:30.415397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:30.415854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.416342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:30.416342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.416342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:30.420427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:30.420618 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:30.420749 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:30.420849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:30.421159 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.421308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.421770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:30.421770 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.421770 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:30.421770 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:30.421770 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.421770 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:30.421770 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.421770 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:30.422508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:30.422875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.422875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:30.422875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.422875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:30.422875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:30.422875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.422875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:30.422875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.422875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.423420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:30.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:30.423912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.423912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:30.423912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.423912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:30.424566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.424849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:30.425123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.425123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:30.425123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:30.425123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.425123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:30.425123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.425123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:30.426419 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:30.426606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:30.426715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:30.426818 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:30.427731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.427939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.428370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:30.428370 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.428370 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:30.428370 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:30.428370 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.428370 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:30.428370 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.428370 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:30.428919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:30.429403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.429403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:30.429403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.429403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:30.429403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:30.429403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.429403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:30.429403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.429403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.429719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:30.430049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:30.430508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:30.430899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.431387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:30.435603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:30.435801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:30.435933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:30.436100 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:30.436462 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.436708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:30.437001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.437480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:30.437480 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.437480 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:30.437480 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:30.437480 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.437480 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:30.437480 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.437480 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.438220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.438780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:30.439213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:30.439663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:30.440154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:30.440154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:30.440154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:30.440154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:30.440154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:30.440154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:35.299862 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:35.300310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:35.300473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:35.301187 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:35.353209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.353426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.354010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:35.354010 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.354010 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:35.354010 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:35.354010 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.354010 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:35.354010 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.354010 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.354928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.355359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:35.356043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.356043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:35.356272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:35.356908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.357246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:35.357246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.357246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:35.409286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:35.409332 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:35.409417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:35.410673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.410884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.411010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.411127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.411243 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.411360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.411485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.411601 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.411758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.411871 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.411984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-f982d743-71d0-4cae-bd42-710875b954a0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.412094 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.412222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.412909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:35.412909 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.412909 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:35.412909 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:35.412909 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.412909 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:35.412909 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.412909 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.413670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.414027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:35.414276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.414276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:35.414384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:35.414758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.415216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:35.415216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.415216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:35.420772 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:35.421022 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:35.421109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:35.421204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:35.421693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.421886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.422468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:35.422468 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.422468 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:35.422468 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:35.422468 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.422468 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:35.422468 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.422468 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:35.423410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:35.423759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.423759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:35.423759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.423759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:35.423759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:35.423759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.423759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:35.423759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.423759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.424232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:35.424474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:35.424725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.424725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:35.424725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.424725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:35.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.426021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:35.426611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.426611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:35.426611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:35.426611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.426611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:35.426611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.426611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:35.426907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:35.427272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:35.427451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:35.427579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:35.427971 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.428157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.428692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:35.428692 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.428692 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:35.428692 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:35.428692 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.428692 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:35.428692 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.428692 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:35.429275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:35.429536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.429536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:35.429536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.429536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:35.429536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:35.429536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.429536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:35.429536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.429536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.429902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:35.430117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:35.430343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.430343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:35.430343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.430343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.430615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:35.430979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:35.431301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.431301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:35.431301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.431301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:35.435784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:35.436184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:35.436327 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:35.436567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:35.437023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.437330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:35.437558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.438091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:35.438091 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.438091 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:35.438091 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:35.438091 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.438091 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:35.438091 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.438091 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.439121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.439418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:35.439766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.439766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:35.440354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:35.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:35.441434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:35.441434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:35.441434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:40.300332 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:40.301236 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:40.301436 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:40.302183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:40.343266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.343759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.344503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:40.344503 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.344503 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:40.344503 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:40.344503 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.344503 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:40.344503 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.344503 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.345489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.345876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:40.346277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.346277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:40.347491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:40.347956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.348495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:40.348495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.348495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:40.409502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:40.410360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:40.410585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:40.414985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.415440 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.415440 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.415707 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.415707 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.415761 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.415926 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.416060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.416188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.416326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.416526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-a58be73f-13d3-46a0-b544-e31553556bb0, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.416676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.417228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:40.417228 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.417228 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:40.417228 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:40.417228 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.417228 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:40.417228 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.417228 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.418209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.418569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:40.418929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.418929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:40.419502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:40.420056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.420531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:40.420531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.420531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:40.421290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:40.421907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:40.421994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:40.422249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:40.427226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.427439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.431207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:40.431207 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.431207 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:40.431207 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:40.431207 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.431207 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:40.431207 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.431207 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:40.432098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:40.432512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.432512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:40.432512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.432512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:40.432512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:40.432512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.432512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:40.432512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.432512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.432775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:40.433097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:40.433482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.433482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:40.433482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.433482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:40.433598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.434060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:40.434530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.434530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:40.434530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:40.434530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.434530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:40.434530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.434530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:40.435123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:40.435391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:40.435495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:40.435635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:40.436108 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.436366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.436712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:40.439185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:40.439185 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.439185 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:40.439185 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:40.439185 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.439185 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:40.439185 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.439185 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:40.439894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:40.440302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.440302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:40.440302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.440302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:40.440302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:40.440302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.440302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:40.440302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.440302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.440687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:40.440992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:40.441336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.441336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:40.441336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.441336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.441585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:40.442058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:40.442528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.442528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:40.442528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.442528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:40.442528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:40.442528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:40.442528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:40.442957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.443198 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:40.443508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.444128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:40.444128 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.444128 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:40.444128 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:40.444128 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.444128 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:40.444128 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.444128 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.444878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.445206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:40.445526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.445526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:40.446828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:40.447271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:40.447839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:40.447839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:40.447839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:45.301231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:45.301664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:45.301920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:45.302592 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:45.336080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.336835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.336927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:45.336927 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.336927 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:45.336927 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:45.336927 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.336927 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:45.336927 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.336927 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.340640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.341253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:45.341508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:45.341831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.342142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:45.342142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:45.342142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.342142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:45.342142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.342142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:45.410364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:45.410660 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:45.410816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:45.411970 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.412223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.412366 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.412534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.412700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.412874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.413032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.413164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.413308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.413467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.413609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.414282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:45.414282 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.414282 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:45.414282 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:45.414282 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.414282 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:45.414282 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.414282 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.415212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.415583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:45.416014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.416014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:45.416278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:45.416746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.417316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:45.417316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.417316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:45.422288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:45.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:45.422676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:45.422787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:45.423168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.423355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.423877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:45.423877 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.423877 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:45.423877 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:45.423877 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.423877 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:45.423877 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.423877 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:45.424557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:45.425055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.425055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:45.425055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.425055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:45.425055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:45.425055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.425055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:45.425055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.425055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.425316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:45.425684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:45.426045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.426443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.426957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:45.435664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:45.436035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:45.437579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:45.437579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:45.437579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.437579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.437839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:45.437839 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.437839 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:45.437839 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:45.437839 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.437839 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:45.437839 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.437839 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:45.438582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:45.439025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.439025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:45.439025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.439025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:45.439025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:45.439025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.439025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:45.439025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.439025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.439707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:45.441558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:45.441995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:45.442350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.442835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:45.443129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:45.443365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:45.443470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:45.443712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:45.444286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.444286 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:45.444649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.448162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:45.448162 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.448162 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:45.448162 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:45.448162 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.448162 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:45.448162 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.448162 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.448967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.449308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:45.449508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.449508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:45.449745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:45.450002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:45.450373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:45.450373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:45.450373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:50.301721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:50.304303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:50.304303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:50.304729 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:50.366664 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.366847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.367355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:50.367355 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.367355 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:50.367355 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:50.367355 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.367355 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:50.367355 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.367355 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.368547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.368919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:50.369600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:50.371230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.371862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:50.371862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:50.371862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.371862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:50.371862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.371862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:50.411065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:50.411361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:50.411502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:50.412849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.413077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.415324 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.415472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.415590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.415712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.415844 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.415964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.416079 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.416173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.416315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.416382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.416976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:50.416976 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.416976 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:50.416976 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:50.416976 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.416976 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:50.416976 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.416976 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.417832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.418106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:50.418306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.418306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:50.418666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:50.419044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.419333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:50.419333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.419333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:50.422695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:50.423048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:50.423173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:50.423297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:50.423719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.423936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.424482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:50.424482 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.424482 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:50.424482 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:50.424482 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.424482 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:50.424482 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.424482 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:50.425238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:50.425685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.425685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:50.425685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.425685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:50.425685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:50.425685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.425685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:50.425685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.425685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.426252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:50.427007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:50.427722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.428389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.428847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:50.436154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:50.436638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:50.436726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:50.436820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:50.437191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.437450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.438173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:50.438173 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.438173 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:50.438173 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:50.438173 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.438173 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:50.438173 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.438173 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:50.439342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:50.439802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:50.440212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:50.440491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:50.441045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.441591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:50.444268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:50.444268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:50.444268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:50.444268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:50.447400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.447400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:50.447803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.449244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:50.449244 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.449244 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:50.449244 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:50.449244 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.449244 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:50.449244 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.449244 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.449910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.450201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:50.450468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.450468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:50.450808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:50.451171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:50.451481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:50.451481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:50.451481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:55.304390 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:55.304884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:55.304884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:55.305344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:55.315339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=243 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:14:55.344744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.344744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.345112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:55.345112 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.345112 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:55.345112 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:55.345112 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.345112 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:55.345112 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.345112 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.345634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.346060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:55.346403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.346403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:55.346479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:55.347120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.347730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:55.347730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.347730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:55.411600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:55.412145 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:55.412301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:55.413966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.414181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.414374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.414552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.414846 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.415132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.415271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.415450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.415636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.415821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.416150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.416320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.416949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:55.416949 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.416949 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:55.416949 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:55.416949 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.416949 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:55.416949 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.416949 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.418543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.419064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:55.419316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.419316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:55.419816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:55.420206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.420685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:55.420685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.420685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:55.423172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:55.423786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:55.424033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:55.424137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:14:55.424501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.424759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.425382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:55.425382 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.425382 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:55.425382 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:55.425382 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.425382 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:55.425382 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.425382 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:55.426319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:55.426959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.426959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:55.426959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.426959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:55.426959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:55.426959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.426959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:55.426959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.426959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.427314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:55.427733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:55.428192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.428192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:55.428192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.428192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:55.429179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.429686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:55.430128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.430128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:55.430128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:55.430128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.430128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:55.430128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.430128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:55.436609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:55.437111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:55.437411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:55.437638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:55.438491 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.438983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.439462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:55.439462 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.439462 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:55.439462 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:55.439462 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.439462 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:55.439462 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.439462 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:14:55.440724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:14:55.441210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.441210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:14:55.441210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.441210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:14:55.441210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:14:55.441210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.441210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:14:55.441210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.441210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.441855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:55.442261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:55.442541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.442541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:55.442541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.442541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.443469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:55.443869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:55.444226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.444226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:55.444226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.444226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:14:55.444330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:14:55.444614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:14:55.444798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:14:55.445034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:14:55.445454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.445701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:14:55.446310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.446914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:14:55.446914 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.446914 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:14:55.446914 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:14:55.446914 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.446914 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:14:55.446914 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.446914 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.448269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.448805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:14:55.449484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:14:55.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:14:55.450569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:14:55.450569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:14:55.450569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:14:55.450569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:14:55.450569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:14:55.450569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:00.304568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:00.306574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:00.306574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:00.307147 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:00.346915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.347143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.347738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:00.347738 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.347738 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:00.347738 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:00.347738 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.347738 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:00.347738 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.347738 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.348427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.348848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:00.349495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:00.350051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.350564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:00.350564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:00.350564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.350564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:00.350564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.350564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:00.412102 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:00.416026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:00.416140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:00.420031 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.420223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.420440 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.420505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.420584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.420741 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.420849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.420954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.421089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.421200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.421339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.421495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.421679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.422291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:00.422291 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.422291 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:00.422291 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:00.422291 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.422291 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:00.422291 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.422291 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.424403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.424711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:00.424910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:00.425164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:00.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:00.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:00.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:00.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:00.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:00.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:00.425449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:00.426937 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.427086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.427556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:00.427556 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.427556 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:00.427556 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:00.427556 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.427556 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:00.427556 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.427556 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:00.428455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:00.428801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.428801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:00.428801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.428801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:00.428801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:00.428801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.428801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:00.428801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.428801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.429206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:00.429430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:00.429771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.429771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:00.429771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.429771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:00.430182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.430736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:00.431181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.431181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:00.431181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:00.431181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.431181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:00.431181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.431181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:00.437014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:00.437241 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:00.437330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:00.437432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:00.438799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.439075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.439484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:00.439484 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.439484 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:00.439484 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:00.439484 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.439484 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:00.439484 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.439484 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:00.440092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:00.440529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.440529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:00.440529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.440529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:00.440529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:00.440529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.440529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:00.440529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.440529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.440822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:00.441136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:00.441506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:00.441926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.442467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:00.444707 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:00.444936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:00.445023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:00.445159 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:00.445538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.445785 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:00.446173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.446706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:00.446706 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.446706 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:00.446706 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:00.446706 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.446706 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:00.446706 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.446706 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.447358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.447708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:00.448031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.448031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:00.448453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:00.448915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:00.449348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:00.449348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:00.449348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:05.306397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:05.306816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:05.306960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:05.307837 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:05.351239 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.351400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.351865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:05.351865 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.351865 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:05.351865 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:05.351865 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.351865 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:05.351865 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.351865 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.352777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.353044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:05.353250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.353250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:05.353499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:05.353830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.354239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:05.354239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.354239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:05.416354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:05.416783 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:05.416903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:05.419536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.419536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.419536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.419536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.419810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.419810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.420203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.420203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.420203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.420397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.420488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.420677 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.420831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4c4ba1d8-3278-4859-bc65-d944b9a9e589, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.420977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.421537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:05.421537 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.421537 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:05.421537 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:05.421537 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.421537 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:05.421537 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.421537 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.422443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.422795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:05.423113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.423113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:05.423608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:05.423968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.424250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:05.424250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.424250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:05.424567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:05.424766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:05.424882 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:05.425021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:05.425311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.425499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.425969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:05.425969 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.425969 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:05.425969 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:05.425969 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.425969 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:05.425969 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.425969 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:05.426701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:05.428296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.428582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.428867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:05.429052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.429350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:05.429648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.429648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:05.429648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:05.429648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.429648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:05.429648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.429648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:05.438298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:05.438539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:05.438658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:05.438761 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:05.439023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.439203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.439591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:05.439591 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.439591 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:05.439591 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:05.439591 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.439591 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:05.439591 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.439591 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:05.440155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:05.440984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:05.441261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.441529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.441744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:05.442068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:05.442359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.442359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:05.442359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.442359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:05.445014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:05.445230 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:05.445351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:05.445439 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:05.445728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.445905 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:05.446212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.446565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:05.446565 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.446565 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:05.446565 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:05.446565 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.446565 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:05.446565 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.446565 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.447127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.447380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:05.447609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.447609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:05.447848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:05.448903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:05.449338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:05.449338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:05.449338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:10.306842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:10.307353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:10.307353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:10.307985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:10.345227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.345500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.346088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:10.346088 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.346088 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:10.346088 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:10.346088 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.346088 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:10.346088 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.346088 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.346902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.347215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:10.347496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.347496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:10.347612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:10.347987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.348428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:10.348428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.348428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:10.416847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:10.417227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:10.417398 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:10.418542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.418744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.418859 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.418971 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.419102 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.419228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.419391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.419520 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.419634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.420014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.420150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.420301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.420461 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4c4ba1d8-3278-4859-bc65-d944b9a9e589, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.420658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.421124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:10.421124 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.421124 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:10.421124 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:10.421124 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.421124 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:10.421124 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.421124 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.421857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.422094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:10.422329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.422329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:10.422800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:10.423089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.423473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:10.423473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.423473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:10.424834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:10.425114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:10.425231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:10.425359 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:10.425729 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.425888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.426306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:10.426306 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.426306 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:10.426306 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:10.426306 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.426306 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:10.426306 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.426306 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:10.426984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:10.427322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.427322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:10.427322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.427322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:10.427322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:10.427322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.427322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:10.427322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.427322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.427805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:10.428422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:10.428712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.428990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.429313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:10.438728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:10.439820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:10.439820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:10.439820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:10.439820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.440115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.441008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.441452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.441770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:10.442001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:10.442297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.442297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:10.442297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.442297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.442427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:10.442788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:10.443187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.443187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:10.443187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.443187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:10.449193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:10.449193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:10.449193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:10.449193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:10.449193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.449193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.449380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.449771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:10.450066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:10.450349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:10.452078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:15.307339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:15.310298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:15.310418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:15.310969 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:15.355118 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.355256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.355879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:15.355879 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.355879 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:15.355879 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:15.355879 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.355879 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:15.355879 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.355879 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.356572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.356930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:15.357249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:15.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.358091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:15.358091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:15.358091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.358091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:15.358091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.358091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:15.417290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:15.417909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:15.418010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:15.419292 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.419489 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.419605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.419735 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.419849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.419949 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.420049 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.420171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.420278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.420377 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.420476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.420575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.420690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-4c4ba1d8-3278-4859-bc65-d944b9a9e589, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.420790 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-624ca9ce-f183-4c1f-b30e-4a762d0b45f5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.420890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.421401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:15.421401 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.421401 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:15.421401 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:15.421401 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.421401 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:15.421401 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.421401 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.422157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.422389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:15.422589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.422589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:15.423181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:15.423461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.423767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:15.423767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.423767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:15.425181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:15.425385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:15.425495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:15.425590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:15.425993 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.426160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.426593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:15.426593 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.426593 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:15.426593 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:15.426593 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.426593 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:15.426593 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.426593 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:15.427097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:15.427352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.427352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:15.427352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.427352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:15.427352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:15.427352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.427352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:15.427352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.427352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.427716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:15.428017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:15.428242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.428242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:15.428242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.428242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:15.428542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.428975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:15.429243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.429243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:15.429243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:15.429243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.429243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:15.429243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.429243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:15.439148 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:15.439450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:15.439568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:15.439705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:15.440162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.440405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.440877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:15.440877 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.440877 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:15.440877 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:15.440877 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.440877 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:15.440877 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.440877 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:15.441527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:15.441792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.441792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:15.441792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.441792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:15.441792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:15.441792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.441792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:15.441792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.441792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.442148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:15.442336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:15.442570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.442570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:15.442570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.442570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.442903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:15.443249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:15.443537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.443537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:15.443537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.443537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:15.445851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:15.446048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:15.446129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:15.446305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:15.446563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.446737 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:15.447017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.447412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:15.447412 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.447412 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:15.447412 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:15.447412 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.447412 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:15.447412 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.447412 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.448063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.448283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:15.448470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.448470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:15.448783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:15.449100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:15.449388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:15.449388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:15.449388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:20.310485 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:20.311344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:20.311344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:20.311643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:20.360067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.360067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.360653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:20.360653 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.360653 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:20.360653 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:20.360653 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.360653 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:20.360653 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.360653 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.362247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.362743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:20.362975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:20.363256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.363609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:20.363609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:20.363609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.363609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:20.363609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.363609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:20.418062 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:20.418469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:20.418581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:20.419980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.420260 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.420429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.420574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.420745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.420879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.421017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.421165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.421304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.421437 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.421568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.421717 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-4c4ba1d8-3278-4859-bc65-d944b9a9e589, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.421846 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=12): DelStaticRoutesCommand(_result=None, lrouter=neutron-624ca9ce-f183-4c1f-b30e-4a762d0b45f5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.421975 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=13): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.422122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.422721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:20.422721 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.422721 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:20.422721 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:20.422721 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.422721 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:20.422721 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.422721 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.423655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.423901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:20.424133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.424133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:20.424729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:20.425183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.425938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:20.425938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.425938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:20.425938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:20.426089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:20.426351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:20.426435 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:20.428202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.428434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.428995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:20.428995 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.428995 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:20.428995 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:20.428995 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.428995 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:20.428995 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.428995 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:20.429711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:20.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:20.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:20.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:20.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:20.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.430121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.430467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:20.430880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:20.431321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.431835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:20.440872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:20.441200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:20.441330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:20.442544 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:20.442544 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.442544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.442899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:20.442899 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.442899 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:20.442899 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:20.442899 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.442899 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:20.442899 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.442899 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:20.443510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:20.443898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.443898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:20.443898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.443898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:20.443898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:20.443898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.443898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:20.443898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.443898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.447677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:20.447974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:20.448310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:20.448616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:20.449005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:20.449302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:20.449302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:20.449302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.449302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:20.449302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.449302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:20.449302 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.449302 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:20.449302 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:20.449456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:20.449725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:20.449963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:20.450278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:20.450607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:20.450607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:20.450607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:20.450607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:20.450607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:20.450607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:25.310888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:25.311418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:25.311418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:25.311912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:25.368125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.368357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.369143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:25.369143 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.369143 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:25.369143 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:25.369143 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.369143 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:25.369143 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.369143 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.058 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.370018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.370317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:25.370548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.370548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:25.372137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:25.372613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.372963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:25.372963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.372963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:25.418560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:25.418855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:25.419813 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:25.420031 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.420226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.420378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.420538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.420728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.420880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.423101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.423101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.423101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.423101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.423101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:25.423970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:25.424325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:25.425081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.425540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:25.426145 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:25.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:25.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:25.426582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:25.429113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.429303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.429780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:25.429780 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.429780 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:25.429780 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:25.429780 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.429780 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:25.429780 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.429780 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:25.430352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:25.430747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.430747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:25.430747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.430747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:25.430747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:25.430747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.430747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:25.430747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.430747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.431204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:25.431761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:25.432090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.432468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.432984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:25.441722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:25.442273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:25.442399 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:25.442547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:25.442988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.443197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.445410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:25.445410 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.445410 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:25.445410 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:25.445410 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.445410 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:25.445410 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.445410 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:25.446038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:25.446378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.446378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:25.446378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.446378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:25.446378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:25.446378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.446378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:25.446378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.446378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.446792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:25.447120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:25.447371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.447371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:25.447371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.447371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:25.447371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.447776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:25.448207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:25.448597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.448597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:25.448597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.448597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:25.448597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:25.448597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:25.448597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:25.448597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.448801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:25.449101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.452233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:25.452233 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.452233 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:25.452233 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:25.452233 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.452233 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:25.452233 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.452233 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.452974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.453272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:25.453509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.453509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:25.453710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:25.454132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:25.454507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:25.454507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:25.454507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:30.313066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:30.313555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:30.313555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:30.315643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:30.349489 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.349769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:30.351770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:30.352158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:30.352429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:30.352783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:30.353155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.353155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:30.353155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.353155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:30.353155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:30.353155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.353155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:30.353155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.353155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:30.419087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:30.419457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:30.419615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:30.420819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.421063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.421232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.421378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.421507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.421655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.421813 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.421948 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.422074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.422207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.422386 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.422459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.422582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:30.424675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:30.425011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:30.425248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.425501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:30.425852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:30.425852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.425852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:30.425852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.425852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:30.425852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:30.425852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.425852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:30.425852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.425852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:30.426081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:30.426311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:30.426476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:30.426647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:30.427050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.427253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.428694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.429073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.429601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:30.429881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:30.430116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.430116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:30.430116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.430116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:30.431061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.431381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:30.431722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.431722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:30.431722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:30.431722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.431722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:30.431722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.431722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:30.442379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:30.443258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:30.443258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:30.443258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:30.443538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.443835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.446435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:30.446839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.447093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.447313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:30.447563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:30.447877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.448212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:30.448212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.448212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:30.448212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:30.448212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.448212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:30.448212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.448212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:30.448212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:30.448610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:30.448812 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:30.449001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:30.449414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.449666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:30.450045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:30.451919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:30.452301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:30.452529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:30.452867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:30.453203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:30.453203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:30.453203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:30.453203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:30.453203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:30.453203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:30.453203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:30.453203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:30.453203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:30.453203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:35.315471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:35.315910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:35.316072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:35.317037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:35.359943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.360252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.360964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:35.360964 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.360964 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:35.360964 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:35.360964 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.360964 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:35.360964 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.360964 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.361905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:35.362491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.362491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:35.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:35.363561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.364059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:35.364059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.364059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:35.420946 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:35.422146 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:35.422253 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:35.423538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.423762 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.423899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.424043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.424208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.424303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.424409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.424516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.424599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.424752 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.424873 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.425013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.425158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.425815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:35.425815 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.425815 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:35.425815 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:35.425815 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.425815 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:35.425815 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.425815 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.426986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.427236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:35.427475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.427475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:35.427475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:35.429552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:35.429868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.430152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:35.430152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.430152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:35.430152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:35.430152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:35.430152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:35.430705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.430772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.431567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:35.431567 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.431567 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:35.431567 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:35.431567 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.431567 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:35.431567 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.431567 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:35.432409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:35.433488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.433773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.433977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:35.434343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.435068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:35.435696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.435696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:35.435696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:35.435696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.435696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:35.435696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.435696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:35.442996 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:35.445329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:35.445329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:35.445329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:35.445670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.445942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.448576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:35.448985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.449260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.449516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:35.449767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:35.450119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.450644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:35.450644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.450644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:35.450644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:35.450644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.450644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:35.450644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.450644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:35.453188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:35.453959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:35.453959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:35.453959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:35.457072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.457314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:35.457579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:35.459641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:35.459933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:35.460138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:35.460407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:35.460877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:35.460877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:35.460877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:35.460877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:35.460877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:35.460877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:35.460877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:35.460877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:35.460877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:35.460877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:40.315936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:40.316465 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:40.316465 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:40.317129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:40.367721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.367920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.368533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:40.368533 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.368533 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:40.368533 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:40.368533 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.368533 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:40.368533 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.368533 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.369613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.369934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:40.370195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.370195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:40.370770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:40.371168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.371528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:40.371528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.371528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:40.422829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:40.423397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:40.423531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:40.424819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.425111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.425309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.425489 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.425678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.425923 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.425923 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.425923 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.426026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.426126 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.426221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.426326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.426418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.426943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:40.426943 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.426943 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:40.426943 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:40.426943 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.426943 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:40.426943 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.426943 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.427715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.428057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:40.428393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.428393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:40.428503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:40.428950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.429423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:40.429423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.429423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:40.429893 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:40.430149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:40.430233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:40.430320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:40.430740 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.430908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.431346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:40.431346 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.431346 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:40.431346 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:40.431346 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.431346 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:40.431346 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.431346 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:40.431923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:40.432302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.432302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:40.432302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.432302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:40.432302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:40.432302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.432302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:40.432302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.432302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.432513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:40.432820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:40.433144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.433144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:40.433144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.433144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:40.433348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.433969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:40.434443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.434443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:40.434443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:40.434443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.434443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:40.434443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.434443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:40.445256 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:40.445565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:40.445709 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:40.446658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:40.446658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.446972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.447639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:40.447639 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.447639 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:40.447639 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:40.447639 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.447639 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:40.447639 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.447639 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:40.448564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:40.449071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.449071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:40.449071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.449071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:40.449071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:40.449071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.449071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:40.449071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.449071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.449629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:40.450015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:40.450413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.450413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:40.450413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.450413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.450861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:40.451221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:40.451559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.451559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:40.451559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.451559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:40.453512 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:40.453809 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:40.454008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:40.454182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:40.454719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.455035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:40.455308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.455858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:40.455858 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.455858 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:40.455858 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:40.455858 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.455858 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:40.455858 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.455858 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.456822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.457116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:40.457372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.457372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:40.457963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:40.458294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:40.458652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:40.458652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:40.458652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:45.316384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:45.316811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:45.316956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:45.317757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:45.538365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:45.544681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:45.546875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:45.546978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:45.565434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.565559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.566005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:45.566005 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.566005 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:45.566005 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:45.566005 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.566005 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:45.566005 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.566005 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.249 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.566602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.566958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:45.567287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:45.567685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.568151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:45.568151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:45.568151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.568151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:45.568151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.568151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:45.568151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:45.568151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:45.568579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.568579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.568743 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.568824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.568935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.569023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.569113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.569202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.569292 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.569392 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.569489 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.569573 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.569682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.570031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:45.570031 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.570031 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:45.570031 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:45.570031 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.570031 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:45.570031 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.570031 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.570629 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.571018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:45.571467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:45.572237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.573037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:45.573037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:45.573037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.573037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:45.573037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.573037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:45.573037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:45.573037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:45.573037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:45.573037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.573330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.573760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.574156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:45.574492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:45.575429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.576211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:45.577267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:45.578018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:45.578694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:45.579521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:45.579884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:45.580203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.580506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.580933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:45.581354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:45.581354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:45.581354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:45.581354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.581354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:45.581354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:45.581525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:45.581915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:45.582261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:45.583376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:45.584495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:45.584495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:45.584495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:45.584495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:45.584495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:45.584495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:45.584495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:45.584495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:45.584495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:45.584495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:50.316941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:50.319430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:50.319541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:50.320154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:50.370189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.370391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.371058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:50.371058 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.371058 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:50.371058 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:50.371058 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.371058 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:50.371058 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.371058 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.371763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.372230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:50.372703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:50.373255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.373899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:50.373899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:50.373899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.373899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:50.373899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.373899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:50.567656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:50.567869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:50.567944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:50.571368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.571886 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.572265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.572642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:50.573026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:50.573475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.574033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:50.574033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:50.574033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.574033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:50.574033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.574033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:50.574033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:50.574033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:50.574033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:50.574033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:50.574700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.575056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.575817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:50.575956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:50.575956 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.575956 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:50.575956 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:50.575956 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.575956 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:50.575956 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.575956 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:50.576490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:50.576964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.576964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:50.576964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.576964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:50.576964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:50.576964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.576964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:50.576964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.576964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.577212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:50.577533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:50.577894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.578272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:50.578763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:50.579098 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:50.579098 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:50.580103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.580103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.580249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:50.580249 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.580249 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:50.580249 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:50.580249 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.580249 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:50.580249 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.580249 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:50.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:50.581441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.581441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:50.581441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.581441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:50.581441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:50.581441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.581441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:50.581441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.581441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.581735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:50.582066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:50.582455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:50.583277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:50.583832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:50.584128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:50.584128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.584965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:50.584965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.585098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:50.585098 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.585098 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:50.585098 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:50.585098 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.585098 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:50.585098 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.585098 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.585694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.586033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:50.586451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:50.586847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:50.587133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:50.587133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:50.587133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:50.587133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:50.587133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:50.587133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:55.319881 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:55.322544 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:55.322735 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:55.323661 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:55.327960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=237 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:15:55.381306 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.381569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.383284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:55.383284 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.383284 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:55.383284 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:55.383284 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.383284 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:55.383284 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.383284 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.384386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:55.384608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.384608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:55.385390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:55.385714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.386087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:55.386087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.386087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:55.568206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:55.569293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:55.569505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:55.574090 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:55.574397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.574702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.574928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.575150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.575346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.575528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.575734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.575917 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.576082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.576224 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.576376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.576516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.576675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.577556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:55.577556 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.577556 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:55.577556 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:55.577556 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.577556 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:55.577556 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.577556 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.578258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.578595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:55.578880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.578880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:55.580636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:55.581086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.581516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:55.581516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.581516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:55.581919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:55.582293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:55.582521 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:55.582742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:15:55.584028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:15:55.585424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.585673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.586273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:55.586273 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.586273 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:55.586273 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:55.586273 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.586273 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:55.586273 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.586273 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:55.587145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:55.587514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.587514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:55.587514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.587514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:55.587514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:55.587514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.587514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:55.587514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.587514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.588078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:55.588382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:55.588655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.588655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:55.588655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.588655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:55.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.589666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:55.590034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.590034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:55.590034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:55.590034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.590034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:55.590034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.590034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:55.590034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:55.590034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:55.590034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:55.592279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.593097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.593705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:55.593705 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.593705 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:55.593705 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:55.593705 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.593705 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:55.593705 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.593705 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:15:55.594678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:15:55.595148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.595148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:15:55.595148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.595148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:15:55.595148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:15:55.595148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.595148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:15:55.595148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.595148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.595648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:55.595852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:55.596184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.596184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:55.596184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.596184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.596542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:55.596921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:55.597315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.597315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:55.597315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.597315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:15:55.597315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:15:55.597315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:15:55.597315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:15:55.597477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.597534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:15:55.597841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.600929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:15:55.600929 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.600929 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:15:55.600929 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:15:55.600929 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.600929 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:15:55.600929 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.600929 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.601952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.602279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:15:55.602519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.602519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:15:55.603018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:15:55.603318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:15:55.603712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:15:55.603712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:15:55.603712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:00.322948 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:00.323406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:00.323406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:00.323968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.059 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:00.384640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.384925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.385141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:00.385337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.385646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:00.569471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:00.569809 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:00.570019 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:00.571949 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.572097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.572209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.572310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.572416 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.572517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.572610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.572735 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.572830 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.572941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.573036 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.573130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.573250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.574953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:00.574953 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.574953 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:00.574953 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:00.574953 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.574953 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:00.574953 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.574953 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.576022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.576299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:00.576568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.576568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:00.578209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:00.578643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.579083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:00.579083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.579083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:00.583168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:00.583668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:00.583826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:00.584751 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:00.585172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.585334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.585778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:00.585778 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.585778 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:00.585778 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:00.585778 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.585778 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:00.585778 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.585778 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:00.586360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:00.586724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.586724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:00.586724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.586724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:00.586724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:00.586724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.586724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:00.586724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.586724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.586973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:00.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:00.587497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.587497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:00.587497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.587497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:00.587773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.588120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:00.588529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.588529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:00.588529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:00.588529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.588529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:00.588529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.588529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:00.589866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:00.590325 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:00.590454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:00.590606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:00.591068 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.591309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.591785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:00.591785 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.591785 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:00.591785 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:00.591785 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.591785 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:00.591785 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.591785 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:00.592408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:00.592764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.592764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:00.592764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.592764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:00.592764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:00.592764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.592764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:00.592764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.592764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.593378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:00.593657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:00.593972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.593972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:00.593972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.593972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.594575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:00.594956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:00.595358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.595358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:00.595358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.595358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:00.597950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:00.598182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:00.598258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:00.598452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:00.598708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.598918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:00.599173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:00.601053 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.601053 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:00.601053 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:00.601053 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.601053 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:00.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.601659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.601922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:00.602128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.602128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:00.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:00.602610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:00.602956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:00.602956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:00.602956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:05.323356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:05.323795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:05.323795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:05.324319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:05.357644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.357895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.358447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:05.358447 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.358447 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:05.358447 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:05.358447 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.358447 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:05.358447 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.358447 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.359409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.359820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:05.360186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.360186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:05.360565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:05.361016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.361516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:05.361516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.361516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:05.569992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:05.570360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:05.570495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:05.572502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.572719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.572870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-c092afd4-834c-4ccb-a655-ffb7b968e37d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.573012 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.573431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.573602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.574019 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.574161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.574304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.574435 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.574563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.574724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=11): DelStaticRoutesCommand(_result=None, lrouter=neutron-7a77fa3e-3278-4e23-8768-c289c5ac448f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.574849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.575891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:05.575891 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.575891 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:05.575891 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:05.575891 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.575891 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:05.575891 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.575891 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.576846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.577054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:05.577283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.577283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:05.577847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:05.578212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.578530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:05.578530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.578530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:05.583843 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:05.584352 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:05.584667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:05.584667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:05.585289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.585289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.585514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:05.585514 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.585514 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:05.585514 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:05.585514 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.585514 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:05.585514 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.585514 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:05.586270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:05.586671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.586671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:05.586671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.586671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:05.586671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:05.586671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.586671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:05.586671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.586671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.587024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:05.587442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:05.588225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.588463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.588865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:05.590492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:05.590852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:05.590852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:05.590967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:05.591202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.591377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.591804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:05.591804 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.591804 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:05.591804 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:05.591804 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.591804 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:05.591804 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.591804 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:05.592593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:05.592962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:05.593338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:05.593777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:05.594223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.594738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:05.598318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:05.598584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:05.598741 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:05.598912 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:05.599262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.599535 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:05.599953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.600463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:05.600463 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.600463 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:05.600463 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:05.600463 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.600463 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:05.600463 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.600463 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.601455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.601774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:05.602006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.602006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:05.603106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:05.603403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:05.603751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:05.603751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:05.603751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:10.324061 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:10.325585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:10.325585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:10.326070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:10.379088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.379088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.379088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:10.379088 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.379088 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:10.379088 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:10.379088 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.379088 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:10.379088 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.379088 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.380842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.384250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:10.385288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:10.387072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.390148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:10.390148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:10.390148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.390148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:10.390148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.390148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:10.570393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:10.570393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:10.570943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:10.575011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.575649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.575649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.575649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.575824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.575901 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.576195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.576195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.576638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.576638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-e94dade4-cb4d-4aaa-9f8b-642d81c63730, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.576638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.577391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:10.577391 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.577391 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:10.577391 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:10.577391 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.577391 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:10.577391 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.577391 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.578553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.580246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:10.580456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:10.582006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.582538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:10.582538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:10.582538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.582538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:10.582538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.582538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:10.584427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:10.584779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:10.584819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:10.584952 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:10.585682 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.585682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.586428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:10.586428 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.586428 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:10.586428 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:10.586428 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.586428 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:10.586428 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.586428 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:10.587196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:10.587769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.587769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:10.587769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.587769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:10.587769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:10.587769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.587769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:10.587769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.587769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.589656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:10.590879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:10.591360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.593180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:10.595084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:10.595796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:10.595796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:10.595796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.595796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.597426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:10.597426 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.597426 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:10.597426 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:10.597426 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.597426 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:10.597426 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.597426 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:10.599700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:10.603336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:10.603894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:10.604434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:10.606466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.608079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:10.609915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:10.609915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:10.609915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:10.610311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:10.610863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.610863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:10.611436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.612382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:10.612382 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.612382 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:10.612382 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:10.612382 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.612382 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:10.612382 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.612382 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.613071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.613928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:10.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:10.615420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:10.617080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:10.617080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:10.617080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:10.617080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:10.617080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:10.617080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:15.324938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:15.325410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:15.325531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:15.326201 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:15.362464 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.362634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.363190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:15.363190 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.363190 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:15.363190 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:15.363190 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.363190 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:15.363190 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.363190 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.363976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.364394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:15.364762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:15.365117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.365547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:15.365547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:15.365547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.365547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:15.365547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.365547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:15.571353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:15.573560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:15.573750 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:15.574892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.575077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.575192 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.575307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.575452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.575557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.575675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.575777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.575876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.575977 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.576077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.578947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:15.578947 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.578947 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:15.578947 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:15.578947 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.578947 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:15.578947 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.578947 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.579799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.580160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:15.580427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.580427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:15.580647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:15.580902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.581188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:15.581188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.581188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:15.584848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:15.585230 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:15.585398 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:15.585558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:15.588148 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.588374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.588891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:15.588891 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.588891 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:15.588891 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:15.588891 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.588891 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:15.588891 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.588891 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:15.589649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:15.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:15.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:15.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:15.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:15.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.590323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:15.590566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:15.590897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.590897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:15.590897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.590897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:15.591230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.591535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:15.591849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.591849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:15.591849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:15.591849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.591849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:15.591849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.591849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:15.591849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:15.591986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:15.591986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:15.592114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:15.592909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.593066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.593583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:15.593583 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.593583 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:15.593583 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:15.593583 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.593583 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:15.593583 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.593583 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:15.594178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:15.594593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.594593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:15.594593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.594593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:15.594593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:15.594593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.594593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:15.594593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.594593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.594959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:15.595265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:15.595570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.595570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:15.595570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.595570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.595838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:15.596122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:15.596449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.596449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:15.596449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.596449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:15.610351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:15.610751 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:15.610904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:15.611087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:15.611519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.611754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:15.612031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.612579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:15.612579 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.612579 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:15.612579 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:15.612579 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.612579 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:15.612579 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.612579 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.613320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:15.613757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.613757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:15.614044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:15.614436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:15.614808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:15.614808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:15.614808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:20.325421 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:20.325983 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:20.325983 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:20.326701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:20.378800 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.379095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.379527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:20.379527 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.379527 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:20.379527 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:20.379527 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.379527 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:20.379527 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.379527 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.380410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.380795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:20.381122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.381122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:20.381501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:20.381996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.382491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:20.382491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.382491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:20.573602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:20.573775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:20.573895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:20.574889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.575080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.575261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bda75896-a81b-4a61-b7cc-7d670ac19fbb, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.575335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.575868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.575868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.575868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.575868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.576060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-3ede9f2c-ece2-48c5-b990-9258c79c0df8, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.576060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=9): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.576234 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=10): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.576389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.576850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:20.576850 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.576850 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:20.576850 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:20.576850 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.576850 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:20.576850 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.576850 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.577640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.578032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:20.578433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.578433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:20.578811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:20.579483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.580067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:20.580067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.580067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:20.585237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:20.585431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:20.585553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:20.585718 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:20.586026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.586233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.586750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:20.586750 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.586750 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:20.586750 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:20.586750 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.586750 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:20.586750 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.586750 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:20.587559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:20.587860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.587860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:20.587860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.587860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:20.587860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:20.587860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.587860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:20.587860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.587860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:20.588701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:20.588942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.588942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:20.588942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.588942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:20.589416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.589721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:20.590122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.590122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:20.590122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:20.590122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.590122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:20.590122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.590122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:20.591918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:20.592068 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:20.592144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:20.592278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:20.592839 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.593162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.593763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:20.593763 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.593763 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:20.593763 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:20.593763 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.593763 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:20.593763 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.593763 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:20.594438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:20.594703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.594703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:20.594703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.594703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:20.594703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:20.594703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.594703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:20.594703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.594703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.595002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:20.595186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:20.595480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.595480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:20.595480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.595480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.595849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:20.596124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:20.596446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.596446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:20.596446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.596446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:20.612501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:20.612501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:20.612501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:20.612501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:20.612501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.612501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:20.612980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:20.614873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:20.615440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:20.615890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:20.616310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:20.616922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:20.616922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:20.616922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:20.616922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:20.616922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:20.616922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:20.616922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:20.616922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:20.616922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:20.616922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:25.328586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:25.328586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:25.328586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:25.328586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:25.376344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.376541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.377260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:25.377260 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.377260 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:25.377260 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:25.377260 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.377260 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:25.377260 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.377260 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.378314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.378648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:25.378967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.378967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:25.379404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:25.379795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.380255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:25.380255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.380255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:25.574037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:25.574434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:25.574537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:25.576222 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.576357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.576486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.576856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.576856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.576856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.577014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.577082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.577589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.577589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.578404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:25.578404 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.578404 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:25.578404 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:25.578404 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.578404 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:25.578404 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.578404 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.579509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.580378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:25.580844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.580844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:25.581341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:25.581784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.582371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:25.582371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.582371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:25.585598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:25.586505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:25.587276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:25.587276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:25.589097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.589097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.590750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:25.590750 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.590750 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:25.590750 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:25.590750 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.590750 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:25.590750 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.590750 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:25.591652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:25.592203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.592592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.592855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:25.594414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.595328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:25.596787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.596787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:25.596787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:25.596787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.596787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:25.596787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.596787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:25.596787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:25.596787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:25.596787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:25.597129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:25.597524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:25.598067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:25.598519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:25.600005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.600871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.601452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:25.611167 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:25.611456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:25.611585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:25.611765 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:25.612139 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.612395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:25.612702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.613426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:25.613426 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.613426 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:25.613426 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:25.613426 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.613426 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:25.613426 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.613426 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.614740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.615304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:25.615668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:25.616938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:25.617413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:25.617413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:25.617413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:25.617413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:25.617413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:25.617413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:30.327269 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:30.327726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:30.327861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:30.328688 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:30.394354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.394612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.395343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:30.395343 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.395343 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:30.395343 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:30.395343 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.395343 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:30.395343 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.395343 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.067 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.396450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.396759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:30.397046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.397046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:30.397111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:30.397408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.397725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:30.397725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.397725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:30.574518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:30.575608 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:30.575754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:30.577336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.577532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.577698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.577825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.577947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.578057 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.578199 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.578261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.578360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.578465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.579048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:30.579048 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.579048 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:30.579048 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:30.579048 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.579048 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:30.579048 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.579048 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.582317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.582764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:30.583189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:30.583679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.584346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:30.584346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:30.584346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.584346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:30.584346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.584346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:30.586596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:30.586926 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:30.587160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:30.587299 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:30.588667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.588667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.588667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:30.588667 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.588667 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:30.588667 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:30.588667 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.588667 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:30.588667 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.588667 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:30.589171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:30.589580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.589580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:30.589580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.589580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:30.589580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:30.589580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.589580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:30.589580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.589580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.589996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:30.590307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:30.590673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.591061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.591553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:30.594684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:30.595022 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:30.595122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:30.595229 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:30.596156 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.596378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.597676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:30.597676 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.597676 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:30.597676 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:30.597676 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.597676 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:30.597676 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.597676 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:30.597839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:30.598120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.598120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:30.598120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.598120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:30.598120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:30.598120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.598120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:30.598120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.598120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.600306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:30.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:30.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:30.601517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.602028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:30.611686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:30.613530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:30.613779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:30.614033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:30.614940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.615157 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:30.615428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.616123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:30.616123 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.616123 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:30.616123 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:30.616123 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.616123 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:30.616123 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.616123 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.617402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.617692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:30.617936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.617936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:30.619010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:30.619328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:30.619668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:30.619668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:30.619668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:35.329804 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:35.330314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:35.330433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:35.331211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:35.361840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.362128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.362399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:35.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.362993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:35.575765 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:35.575903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:35.575947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:35.576915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.577103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.577237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.577345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.577456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.577558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.577668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.577780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.577877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.577975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.578570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:35.578570 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.578570 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:35.578570 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:35.578570 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.578570 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:35.578570 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.578570 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.579272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.579564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:35.579917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.579917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:35.579976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:35.580348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.580807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:35.580807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.580807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:35.587021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:35.588215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:35.588215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:35.588215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:35.588909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.592856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.593823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:35.593823 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.593823 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:35.593823 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:35.593823 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.593823 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:35.593823 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.593823 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:35.596006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:35.596344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.596344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:35.596344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.596344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:35.596344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:35.596344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.596344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:35.596344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.596344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:35.596344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.598066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:35.598354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:35.598600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.598600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:35.598600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.598600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:35.603000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.603365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:35.603672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.603672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:35.603672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:35.603672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.603672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:35.603672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.603672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:35.603672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:35.603672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:35.603672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:35.604152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.604455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.605453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:35.605453 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.605453 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:35.605453 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:35.605453 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.605453 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:35.605453 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.605453 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:35.606518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:35.606942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.606942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:35.606942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.606942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:35.606942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:35.606942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.606942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:35.606942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.606942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.607462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:35.607777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:35.608107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.608107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:35.608107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.608107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.608600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:35.609040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:35.609487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.609487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:35.609487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.609487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:35.613654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:35.615348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:35.615652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:35.616076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:35.616493 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.617527 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:35.618224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.618973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:35.618973 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.618973 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:35.618973 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:35.618973 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.618973 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:35.618973 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.618973 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.620097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.620606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:35.621021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.621021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:35.621175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:35.621692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:35.622353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:35.622353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:35.622353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:40.331825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:40.331825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:40.331825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:40.331825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:40.363706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.363864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.364329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:40.364329 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.364329 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:40.364329 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:40.364329 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.364329 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:40.364329 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.364329 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.364949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.365322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:40.365754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:40.366181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.366799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:40.366799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:40.366799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.366799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:40.366799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.366799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:40.576051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:40.576310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:40.576410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:40.577388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.577541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.577685 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.577796 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.577909 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.578015 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.578120 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.578226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.578331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.578440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.578941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:40.578941 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.578941 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:40.578941 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:40.578941 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.578941 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:40.578941 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.578941 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.579546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.579835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:40.580086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.580086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:40.580398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:40.580777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.581133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:40.581133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.581133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:40.587416 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:40.587686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:40.587772 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:40.587860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:40.589182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.589995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.590484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:40.590484 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.590484 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:40.590484 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:40.590484 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.590484 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:40.590484 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.590484 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:40.591164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:40.591463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.591463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:40.591463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.591463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:40.591463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:40.591463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.591463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:40.591463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.591463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.592006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:40.592243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:40.592597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.592597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:40.592597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.592597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:40.593110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.593430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:40.593815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.593815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:40.593815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:40.593815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.593815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:40.593815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.593815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:40.603404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:40.603730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:40.603861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:40.604020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:40.605016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.605206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.605673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:40.605673 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.605673 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:40.605673 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:40.605673 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.605673 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:40.605673 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.605673 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:40.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:40.606794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.606794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:40.606794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.606794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:40.606794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:40.606794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.606794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:40.606794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.606794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.607295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:40.607682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:40.608104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.608104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:40.608104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.608104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.608418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:40.608798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:40.609099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.609099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:40.609099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.609099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:40.614083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:40.614377 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:40.614498 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:40.614726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:40.616640 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.616860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:40.617127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.619587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:40.619587 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.619587 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:40.619587 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:40.619587 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.619587 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:40.619587 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.619587 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.620335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.620601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:40.620889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.620889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:40.621169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:40.621510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:40.621914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:40.621914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:40.621914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:45.333018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:45.333429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:45.333565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:45.334430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:45.375604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.376117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.377237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:45.377237 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.377237 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:45.377237 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:45.377237 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.377237 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:45.377237 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.377237 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.381705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.382581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:45.382987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:45.383693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.384317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:45.384317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:45.384317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.384317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:45.384317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.384317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:45.577389 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:45.578947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:45.579054 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:45.580040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.580109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.580251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.580363 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.580499 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.580615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.580779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.580878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.580991 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.581113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.587811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:45.591832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:45.591832 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.591832 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:45.591832 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:45.591832 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.591832 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:45.591832 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.591832 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.592668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.593081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:45.593469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:45.593918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.594463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:45.594463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:45.594463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.594463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:45.594463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.594463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:45.594463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:45.594463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:45.594463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:45.596246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.596479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.599920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:45.600863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.601314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.601710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:45.602082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.602082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:45.603580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.604042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:45.604560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.604560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:45.604560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:45.604560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.604560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:45.604560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.604560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:45.604560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:45.604560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:45.604560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:45.604875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:45.604875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.604875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.605056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:45.605056 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.605056 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:45.605056 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:45.605056 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.605056 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:45.605056 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.605056 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:45.605559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:45.605983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.605983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:45.605983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.605983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:45.605983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:45.605983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.605983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:45.605983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.605983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.606250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:45.606593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:45.606970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.606970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:45.606970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.606970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.607098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:45.607558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:45.608081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.608081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:45.608081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.608081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:45.614475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:45.614754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:45.614860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:45.615010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:45.615254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.615467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:45.615700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.616061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:45.616061 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.616061 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:45.616061 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:45.616061 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.616061 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:45.616061 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.616061 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.616609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.616975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:45.617330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:45.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:45.618338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:45.618338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:45.618338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:45.618338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:45.618338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:45.618338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:50.333459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:50.333888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:50.333934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:50.334500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:50.361980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.362144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:50.362618 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.362618 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:50.362618 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:50.362618 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.362618 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:50.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.362618 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.363323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.363571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:50.363806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.363806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:50.364248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:50.364539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.364946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:50.364946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.364946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:50.579182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:50.579470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:50.579591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:50.580518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.580676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.580907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.581025 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.581122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.581223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.581345 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.581463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.581590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.581721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.582195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:50.582195 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.582195 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:50.582195 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:50.582195 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.582195 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:50.582195 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.582195 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.582890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.583173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:50.583434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.583434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:50.583687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:50.584155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.584704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:50.584704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.584704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:50.593778 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:50.594009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:50.594118 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:50.594255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:50.594548 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.594753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.595246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:50.595246 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.595246 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:50.595246 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:50.595246 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.595246 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:50.595246 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.595246 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:50.595859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:50.596151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.596151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:50.596151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.596151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:50.596151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:50.596151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.596151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:50.596151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.596151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.596687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:50.596987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:50.597370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.597370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:50.597370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.597370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:50.597646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.597984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:50.598352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.598352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:50.598352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:50.598352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.598352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:50.598352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.598352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:50.604064 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:50.604312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:50.604411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:50.604521 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:50.605134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.605134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.605480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:50.605480 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.605480 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:50.605480 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:50.605480 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.605480 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:50.605480 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.605480 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:50.606128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:50.606444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.606444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:50.606444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.606444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:50.606444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:50.606444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.606444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:50.606444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.606444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.606903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:50.607161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:50.607459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.607459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:50.607459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.607459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.607733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:50.608106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:50.608563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.608563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:50.608563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.608563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:50.614845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:50.615101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:50.615223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:50.615399 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:50.615910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.616152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:50.616450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.616998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:50.616998 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.616998 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:50.616998 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:50.616998 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.616998 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:50.616998 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.616998 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.617810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:50.618541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.618541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:50.618850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:50.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:50.619823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:50.619823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:50.619823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:55.333903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:55.334429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:55.334429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:55.335174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:55.341217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=241 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:16:55.371535 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.371857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.372469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:55.372469 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.372469 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:55.372469 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:55.372469 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.372469 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:55.372469 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.372469 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.373153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.373570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:55.373979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:55.374468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.375205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:55.375205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:55.375205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.375205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:55.375205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.375205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:55.579607 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:55.579904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:55.580001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:55.581134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.581759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.581759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.582000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.582000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.582184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.582342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.582506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.582680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.582834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:55.583610 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.583610 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:55.583610 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:55.583610 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.583610 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:55.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.584393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.584683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:55.584907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.584907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:55.585327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:55.585608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.586010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:55.586010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.586010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:55.594245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:55.594674 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:55.594818 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:55.594945 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:16:55.595400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.595652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.596292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:55.596292 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.596292 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:55.596292 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:55.596292 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.596292 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:55.596292 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.596292 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:55.597244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:55.597566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.597566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:55.597566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.597566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:55.597566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:55.597566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.597566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:55.597566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.597566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.598052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:55.598385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:55.598616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.598616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:55.598616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.598616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:55.599201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.599908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:55.600613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.600613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:55.600613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:55.600613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.600613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:55.600613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.600613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:55.604433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:55.604797 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:55.604908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:55.605040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:55.606326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.606520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.607020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:55.607020 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.607020 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:55.607020 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:55.607020 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.607020 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:55.607020 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.607020 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:16:55.607899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:16:55.608262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.608262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:16:55.608262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.608262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:16:55.608262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:16:55.608262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.608262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:16:55.608262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.608262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.608809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:55.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:55.609757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.609757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:55.609757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.609757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.609970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:55.610289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:55.610582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.610582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:55.610582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.610582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:16:55.615290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:16:55.615567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:16:55.615710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:16:55.616791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:16:55.616791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.616791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:16:55.617082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:16:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:16:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:16:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:16:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.618585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.618916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:16:55.619177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.619177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:16:55.619702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:16:55.620039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:16:55.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:16:55.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:16:55.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:00.334335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:00.334927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:00.334927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:00.335261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:00.365976 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.366136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.366561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:00.366561 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.366561 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:00.366561 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:00.366561 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.366561 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:00.366561 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.366561 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.367184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.367455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:00.367709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.367709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:00.367932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:00.368298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.368616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:00.368616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.368616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:00.580003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:00.580213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:00.580296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:00.581111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.581244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-e96bfc13-6b39-40c6-ba02-1794c1c7f669, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.581364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.581516 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.581668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.581814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.581963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.582264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.582264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e68e2d87-4bfd-4c73-960a-b6d3193d2f9d, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.582388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.582757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:00.582757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.582757 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:00.582757 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:00.582757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.582757 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:00.582757 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.582757 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.583610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.584052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:00.584349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:00.585312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.585724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:00.585724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:00.585724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.585724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:00.585724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.585724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:00.594760 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:00.594957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:00.595077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:00.595208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:00.595530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.595731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.596172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:00.596172 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.596172 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:00.596172 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:00.596172 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.596172 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:00.596172 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.596172 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:00.596847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:00.597377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.597377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:00.597377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.597377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:00.597377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:00.597377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.597377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:00.597377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.597377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:00.598191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:00.598586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.598586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:00.598586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.598586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:00.598944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:00.599571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.599571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:00.599571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:00.599571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.599571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:00.599571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.599571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:00.604817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:00.605011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:00.605086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:00.605190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:00.605460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.605696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.606139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:00.606139 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.606139 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:00.606139 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:00.606139 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.606139 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:00.606139 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.606139 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:00.606832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:00.607107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.607107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:00.607107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.607107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:00.607107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:00.607107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.607107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:00.607107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.607107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:00.607836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:00.608092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.608092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:00.608092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.608092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.608339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:00.608615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:00.608951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.608951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:00.608951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.608951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:00.615747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:00.616826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:00.616826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:00.616826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:00.616826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.616964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:00.617153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.617733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:00.617733 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.617733 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:00.617733 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:00.617733 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.617733 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:00.617733 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.617733 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.618817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:00.619062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.619062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:00.619544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:00.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:00.620208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:00.620208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:00.620208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:05.336019 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:05.336510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:05.336510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:05.338116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:05.401683 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.401978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.402705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:05.402705 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.402705 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:05.402705 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:05.402705 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.402705 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:05.402705 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.402705 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.066 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.403587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.403925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:05.404186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.404186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:05.404438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:05.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.405351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:05.405351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.405351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:05.580427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:05.581947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:05.582032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:05.583011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.583252 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.583412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.584125 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.584277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.584414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.584533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.584671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.585411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:05.585411 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.585411 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:05.585411 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:05.585411 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.585411 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:05.585411 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.585411 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.586203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.586578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:05.587011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.587011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:05.588915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:05.589249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.589618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:05.589618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.589618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:05.595130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:05.595556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:05.595702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:05.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:05.596258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.596447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.598083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:05.598083 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.598083 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:05.598083 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:05.598083 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.598083 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:05.598083 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.598083 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:05.598837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:05.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:05.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:05.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:05.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:05.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.599274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.599834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:05.600095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:05.600332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.600332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:05.600332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.600332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:05.605920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.606439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:05.607006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.607006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:05.607006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:05.607006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.607006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:05.607006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.607006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:05.607006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:05.607006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:05.607338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:05.607338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:05.607720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.607937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.608415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:05.608415 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.608415 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:05.608415 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:05.608415 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.608415 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:05.608415 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.608415 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:05.609113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:05.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:05.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:05.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:05.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:05.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.609480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.609998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:05.610283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:05.610666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.610666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:05.610666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.610666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:05.611563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:05.612023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.612023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:05.612023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.612023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:05.616830 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:05.620446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:05.620446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:05.620446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:05.620757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.620969 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:05.621249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.621660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:05.621660 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.621660 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:05.621660 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:05.621660 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.621660 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:05.621660 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.621660 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.622264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.622664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:05.623209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:05.623706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:05.624282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:05.624282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:05.624282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:05.624282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:05.624282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:05.624282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:10.336648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:10.337184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:10.337184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:10.337798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:10.371328 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.371599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.372191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:10.372191 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.372191 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:10.372191 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:10.372191 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.372191 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:10.372191 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.372191 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.372921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.373181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:10.373380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.373380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:10.373602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:10.373894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.374283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:10.374283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.374283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:10.582224 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:10.582802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:10.582979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:10.584255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.584504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.584812 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.584964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.585107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.585246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.585381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.585516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.586229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:10.586229 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.586229 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:10.586229 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:10.586229 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.586229 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:10.586229 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.586229 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.587079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.587297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:10.587534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.587534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:10.587863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:10.588300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.588889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:10.588889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.588889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:10.595589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:10.595974 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:10.596080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:10.596184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:10.596553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.596777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.597288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:10.597288 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.597288 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:10.597288 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:10.597288 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.597288 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:10.597288 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.597288 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:10.597872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:10.598149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.598149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:10.598149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.598149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:10.598149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:10.598149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.598149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:10.598149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.598149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.598443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:10.598725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:10.599019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.599019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:10.599019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.599019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:10.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.599413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:10.599741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.599741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:10.599741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:10.599741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.599741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:10.599741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.599741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:10.607061 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:10.607212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:10.607342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:10.607448 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:10.607852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.608091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.608596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:10.608596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.608596 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:10.608596 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:10.608596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.608596 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:10.608596 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.608596 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:10.609151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:10.609540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.609540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:10.609540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.609540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:10.609540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:10.609540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.609540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:10.609540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.609540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.609785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:10.609987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:10.610244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.610244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:10.610244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.610244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.610525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:10.610890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:10.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:10.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:10.620144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:10.620324 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:10.620397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:10.620553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:10.620983 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.621219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:10.621416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.621899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:10.621899 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.621899 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:10.621899 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:10.621899 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.621899 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:10.621899 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.621899 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.622463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.622704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:10.623038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.623038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:10.623117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:10.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:10.624145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:10.624145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:10.624145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:15.337012 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:15.339423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:15.339423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:15.339423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:15.393388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.393606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.394030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:15.394030 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.394030 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:15.394030 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:15.394030 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.394030 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:15.394030 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.394030 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.056 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.394720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.395036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:15.395311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.395311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:15.395850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:15.396397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.396827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:15.396827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.396827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:15.582820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:15.584027 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:15.584027 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:15.584271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.584457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.584602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.584826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.584970 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.585113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.585292 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.585410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.585967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:15.585967 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.585967 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:15.585967 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:15.585967 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.585967 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:15.585967 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.585967 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.587896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.588185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:15.588384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:15.588681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.588991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:15.588991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:15.588991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.588991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:15.588991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.588991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:15.596116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:15.596456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:15.596652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:15.596845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:15.597770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.597970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.601025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:15.601562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.601939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.602265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:15.602603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:15.602992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:15.603504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.603504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:15.603504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.603504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:15.603504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:15.603504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.603504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:15.603504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.603504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:15.607598 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:15.607928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:15.608096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:15.608312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:15.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.609813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.612218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:15.612846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.613260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.613700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:15.614121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:15.614637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:15.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:15.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:15.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:15.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.617848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:15.620556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:15.620993 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:15.621302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:15.621302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:15.621613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.621853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:15.624042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:15.624452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:15.624701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:15.624960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:15.625285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:15.625285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:15.625285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:15.625285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:15.625285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:15.625285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:15.625285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:15.625285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:15.625285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:15.625285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:20.338116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:20.338551 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:20.338712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:20.339288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:20.377274 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.377486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.378095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:20.378095 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.378095 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:20.378095 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:20.378095 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.378095 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:20.378095 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.378095 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.378917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.379184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:20.379489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.379489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:20.379566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:20.379892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.381397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:20.381397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.381397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:20.583746 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:20.584032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:20.584328 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:20.586058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.586058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.586058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.586700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.586700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.586700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.586700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.586700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.586897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.587478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:20.587478 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.587478 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:20.587478 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:20.587478 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.587478 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:20.587478 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.587478 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.588541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.588866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:20.589201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.589201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:20.589610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:20.590011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.590462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:20.590462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.590462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:20.596583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:20.596911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:20.597045 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:20.597185 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:20.598092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.598092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.600052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:20.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.601077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.601431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:20.601836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:20.602282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:20.602798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.602798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:20.602798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.602798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:20.602798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:20.602798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.602798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:20.602798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.602798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:20.608417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:20.608417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:20.608872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:20.608872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:20.609606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.609771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.610672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:20.610672 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.610672 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:20.610672 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:20.610672 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.610672 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:20.610672 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.610672 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:20.611557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:20.611979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.611979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:20.611979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.611979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:20.611979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:20.611979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.611979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:20.611979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.611979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.612608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:20.612905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:20.613207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.613207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:20.613207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.613207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.613841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:20.614282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:20.614771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.614771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:20.614771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.614771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:20.621109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:20.621829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:20.621829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:20.621829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:20.622211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.622660 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:20.622929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.623634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:20.623634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.623634 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:20.623634 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:20.623634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.623634 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:20.623634 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.623634 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.624529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.624906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:20.625288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.625288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:20.626360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:20.626960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:20.627612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:20.627612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:20.627612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:25.338614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:25.339504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:25.340060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:25.343530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:25.427927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.427927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.429144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:25.429144 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.429144 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:25.429144 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:25.429144 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.429144 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:25.429144 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.429144 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.089 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.432053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.432331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:25.432605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.432605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:25.437643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:25.438409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.439225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:25.439225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.439225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:25.583906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:25.584343 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:25.584442 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:25.585377 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.585554 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.585685 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.585794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.585918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.586018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.586134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.586237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.586332 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.586435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.586931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:25.586931 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.586931 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:25.586931 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:25.586931 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.586931 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:25.586931 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.586931 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.587702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.587988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:25.588237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.588237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:25.588426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:25.588883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.589369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:25.589369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.589369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:25.596988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:25.597441 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:25.597794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:25.597794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:25.598294 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.598516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.599142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:25.599142 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.599142 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:25.599142 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:25.599142 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.599142 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:25.599142 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.599142 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:25.600079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:25.600372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.600372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:25.600372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.600372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:25.600372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:25.600372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.600372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:25.600372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.600372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.601053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:25.601335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:25.601557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.601557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:25.601557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.601557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:25.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.602567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:25.602871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.602871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:25.602871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:25.602871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.602871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:25.602871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.602871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:25.608840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:25.609162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:25.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:25.609517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:25.609942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.610181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.610751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:25.610751 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.610751 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:25.610751 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:25.610751 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.610751 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:25.610751 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.610751 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:25.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:25.611857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.611857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:25.611857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.611857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:25.611857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:25.611857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.611857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:25.611857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.611857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.612477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:25.612716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:25.612968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.612968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:25.612968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.612968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.613563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:25.613881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:25.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:25.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:25.621506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:25.621815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:25.621918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:25.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:25.622470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.622737 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:25.623017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.623585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:25.623585 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.623585 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:25.623585 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:25.623585 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.623585 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:25.623585 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.623585 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.624234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:25.624952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:25.625389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:25.625973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:25.625973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:25.625973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:25.625973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:25.625973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:25.625973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:30.339058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:30.339655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:30.339655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:30.340925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:30.383754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.383988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.384556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:30.384556 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.384556 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:30.384556 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:30.384556 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.384556 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:30.384556 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.384556 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.385547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.385847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:30.386111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.386111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:30.386586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:30.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.387220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:30.387220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.387220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:30.584426 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:30.585067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:30.585067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:30.585987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.586279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.586496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.586706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.586884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.587111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.587273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.587439 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.587599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=8): DelStaticRoutesCommand(_result=None, lrouter=neutron-e921fb53-d55b-4b05-a0ac-2d1289e91b6c, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.587771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.588404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:30.588404 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.588404 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:30.588404 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:30.588404 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.588404 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:30.588404 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.588404 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.589116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.589484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:30.589851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.589851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:30.589929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:30.590326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.590849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:30.590849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.590849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:30.597475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:30.597852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:30.597953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:30.598049 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:30.598400 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.598582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.600805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:30.600805 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.600805 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:30.600805 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:30.600805 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.600805 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:30.600805 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.600805 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:30.601374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:30.601653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.601653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:30.601653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.601653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:30.601653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:30.601653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.601653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:30.601653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.601653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.601979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:30.602189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:30.602434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.602434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:30.602434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.602434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:30.602916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.603198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:30.603548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.603548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:30.603548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:30.603548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.603548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:30.603548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.603548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:30.609220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:30.609559 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:30.609678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:30.609787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:30.610081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.610270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.610683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:30.610683 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.610683 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:30.610683 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:30.610683 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.610683 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:30.610683 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.610683 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:30.611192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:30.611448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.611448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:30.611448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.611448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:30.611448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:30.611448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.611448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:30.611448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.611448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.611858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:30.612064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:30.612306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.612306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:30.612306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.612306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.612540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:30.612906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:30.613205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.613205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:30.613205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.613205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:30.621918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:30.622106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:30.622199 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:30.622365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:30.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.622822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:30.623060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.623446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:30.623446 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.623446 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:30.623446 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:30.623446 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.623446 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:30.623446 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.623446 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.624061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.624423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:30.624777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:30.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:30.625705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:30.625705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:30.625705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:30.625705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:30.625705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:30.625705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:35.339452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:35.339967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:35.340078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:35.340789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:35.374655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.374655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.375049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:35.375049 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.375049 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:35.375049 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:35.375049 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.375049 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:35.375049 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.375049 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.375786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.376003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:35.376217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.376217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:35.376529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:35.376853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.377151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:35.377151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.377151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:35.584924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:35.585266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:35.585370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:35.586678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.586678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.586678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-bf65d767-60ad-4e00-ae84-b54182b65a73, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.586678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.586678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.586896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.586924 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.587059 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.587218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.587758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:35.587758 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.587758 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:35.587758 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:35.587758 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.587758 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:35.587758 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.587758 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.588348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.588581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:35.588825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.588825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:35.589057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:35.589438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.589830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:35.589830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.589830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:35.597986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:35.598251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:35.598356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:35.598430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:35.598880 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.599046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.599691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:35.599691 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.599691 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:35.599691 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:35.599691 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.599691 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:35.599691 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.599691 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:35.600648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:35.600972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.600972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:35.600972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.600972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:35.600972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:35.600972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.600972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:35.600972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.600972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.601639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:35.602062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:35.602383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.602383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:35.602383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.602383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:35.603254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.603529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:35.604114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.604114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:35.604114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:35.604114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.604114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:35.604114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.604114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:35.609739 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:35.610182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:35.610315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:35.610486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:35.610906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.611132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.611599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:35.611599 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.611599 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:35.611599 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:35.611599 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.611599 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:35.611599 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.611599 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:35.612505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:35.612896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.612896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:35.612896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.612896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:35.612896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:35.612896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.612896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:35.612896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.612896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.613560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:35.613815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:35.614144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.614144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:35.614144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.614144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.614733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:35.615056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:35.615428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.615428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:35.615428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.615428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:35.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:35.622472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:35.622554 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:35.622740 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:35.623061 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.623393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:35.623906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.624485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:35.624485 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.624485 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:35.624485 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:35.624485 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.624485 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:35.624485 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.624485 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.625193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.625443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:35.625831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.625831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:35.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:35.626332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:35.626873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:35.626873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:35.626873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:40.340428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:40.340838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:40.340838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:40.341376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:40.372423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.372692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.373286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:40.373286 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.373286 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:40.373286 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:40.373286 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.373286 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:40.373286 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.373286 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.374010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.374320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:40.374781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:40.375177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.375616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:40.375616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:40.375616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.375616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:40.375616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.375616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:40.585370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:40.585665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:40.586915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:40.586915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.586915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.587017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.587164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.587295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.587419 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.587545 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.587699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.587837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.588702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:40.588702 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.588702 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:40.588702 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:40.588702 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.588702 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:40.588702 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.588702 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.589225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.589485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:40.589724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.589724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:40.590182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:40.590538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.590885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:40.590885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.590885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:40.598329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:40.598559 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:40.598658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:40.598784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:40.599107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.599301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.599793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:40.599793 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.599793 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:40.599793 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:40.599793 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.599793 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:40.599793 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.599793 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:40.600474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:40.600812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.600812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:40.600812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.600812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:40.600812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:40.600812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.600812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:40.600812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.600812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.601308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:40.601712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:40.602002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.602002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:40.602002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.602002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:40.602233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.602520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:40.602848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.602848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:40.602848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:40.602848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.602848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:40.602848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.602848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:40.610143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:40.610321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:40.610401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:40.610497 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:40.610797 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.611021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:40.611465 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.611465 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:40.611465 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:40.611465 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.611465 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:40.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:40.612119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:40.612598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.612598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:40.612598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.612598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:40.612598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:40.612598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.612598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:40.612598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.612598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.613043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:40.613395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:40.613833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.613833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:40.613833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.613833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.614267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:40.614599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:40.614984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.614984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:40.614984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.614984 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:40.622577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:40.622833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:40.622950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:40.623115 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:40.623423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.623672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:40.623933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.624406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:40.624406 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.624406 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:40.624406 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:40.624406 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.624406 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:40.624406 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.624406 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.625481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.625911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:40.626284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.626284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:40.626505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:40.626865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:40.627177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:40.627177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:40.627177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:45.340852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:45.342582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:45.342582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:45.342582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:45.368318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.368484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.368902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:45.368902 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.368902 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:45.368902 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:45.368902 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.368902 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:45.368902 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.368902 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.369483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.369790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:45.370028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.370028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:45.370148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:45.370487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.370874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:45.370874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.370874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:45.585786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:45.585998 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:45.586092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:45.586917 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.587088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.587233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.587378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.587504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.587601 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.587711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.587803 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.587894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.588302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:45.588302 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.588302 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:45.588302 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:45.588302 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.588302 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:45.588302 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.588302 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.589041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.589347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:45.589771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.589771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:45.590065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:45.590512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.591054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:45.591054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.591054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:45.598561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:45.598733 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:45.598838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:45.598953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:45.599194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.599368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.599778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:45.599778 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.599778 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:45.599778 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:45.599778 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.599778 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:45.599778 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.599778 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:45.600357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:45.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:45.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:45.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:45.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:45.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.600677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.601117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:45.601320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:45.601560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.601560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:45.601560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.601560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:45.601987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:45.602548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.602548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:45.602548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:45.602548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.602548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:45.602548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.602548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:45.610372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:45.610541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:45.610660 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:45.610791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:45.611022 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.611218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.611613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:45.611613 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.611613 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:45.611613 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:45.611613 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.611613 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:45.611613 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.611613 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:45.612175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:45.612426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.612426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:45.612426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.612426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:45.612426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:45.612426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.612426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:45.612426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.612426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.612942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:45.613130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:45.613408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.613408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:45.613408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.613408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.613849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:45.614139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:45.614432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.614432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:45.614432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.614432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:45.622843 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:45.622986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:45.623063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:45.623173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:45.623376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.623547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:45.623751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.624077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:45.624077 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.624077 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:45.624077 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:45.624077 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.624077 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:45.624077 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.624077 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.624676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.625026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:45.625406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:45.625855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:45.626358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:45.626358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:45.626358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:45.626358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:45.626358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:45.626358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:50.341569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:50.342074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:50.342074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:50.342906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:50.371427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.371657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.372169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:50.372169 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.372169 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:50.372169 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:50.372169 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.372169 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:50.372169 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.372169 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.372876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.373287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:50.373653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:50.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.374595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:50.374595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:50.374595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.374595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:50.374595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.374595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:50.586172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:50.586473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:50.586575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:50.587385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.587542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.587679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.587793 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.587906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.588452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.588452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.588452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.588452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.588887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:50.588887 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.588887 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:50.588887 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:50.588887 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.588887 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:50.588887 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.588887 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.589579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.590843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:50.591105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:50.591354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.591663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:50.591663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:50.591663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.591663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:50.591663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.591663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:50.598903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:50.598971 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:50.599116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:50.599177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:50.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.600575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.600881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:50.600881 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.600881 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:50.600881 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:50.600881 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.600881 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:50.600881 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.600881 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:50.601630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:50.602203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.602203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:50.602203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.602203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:50.602203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:50.602203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.602203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:50.602203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.602203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.602547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:50.603687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:50.603940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.604257 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.604709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:50.611451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:50.611451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:50.611451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:50.611451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:50.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.612268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:50.612268 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.612268 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:50.612268 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:50.612268 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.612268 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:50.612268 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.612268 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:50.613549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:50.614184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:50.614961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:50.615409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:50.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.615911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:50.623234 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:50.623366 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:50.623455 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:50.623604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:50.624045 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.624218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:50.624421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.624955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:50.624955 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.624955 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:50.624955 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:50.624955 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.624955 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:50.624955 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.624955 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.625752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.626080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:50.626415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:50.626811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:50.627299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:50.627299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:50.627299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:50.627299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:50.627299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:50.627299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:55.341923 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:55.342330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:55.342330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:55.342788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:55.343801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=239 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:17:55.375649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.375649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.375840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:55.375840 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.375840 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:55.375840 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:55.375840 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.375840 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:55.375840 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.375840 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.376431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.376697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:55.376885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.376885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:55.377452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:55.377760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.378073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:55.378073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.378073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:55.586612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:55.586863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:55.586947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:55.587910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.588139 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.588254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.588359 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.588459 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.588555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.588671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.588768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.588869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.589363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:55.589363 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.589363 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:55.589363 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:55.589363 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.589363 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:55.589363 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.589363 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.590155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.590380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:55.590711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.590711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:55.590844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:55.591118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.591466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:55.591466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.591466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:55.599035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:55.599203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:55.599287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:55.599374 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:17:55.599822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.599998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.600438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:55.600438 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.600438 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:55.600438 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:55.600438 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.600438 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:55.600438 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.600438 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:55.600947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:55.601248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.601248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:55.601248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.601248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:55.601248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:55.601248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.601248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:55.601248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.601248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.601509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:55.601775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:55.602012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.602012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:55.602012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.602012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:55.602169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.602436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:55.602752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.602752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:55.602752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:55.602752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.602752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:55.602752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.602752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:55.610957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:55.611135 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:55.611217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:55.611332 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:55.611827 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.612054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.612871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:55.612871 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.612871 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:55.612871 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:55.612871 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.612871 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:55.612871 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.612871 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:17:55.613475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:17:55.613767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.613767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:17:55.613767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.613767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:17:55.613767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:17:55.613767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.613767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:17:55.613767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.613767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.614112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:55.614328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:55.614550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.614550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:55.614550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.614550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.614815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:55.615146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:55.615500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.615500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:55.615500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.615500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:55.623798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:55.624338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:55.624745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:55.624913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:17:55.625532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.625826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:55.626125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.626817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:55.626817 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.626817 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:55.626817 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:55.626817 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.626817 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:55.626817 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.626817 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.627730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:55.627954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.627954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:17:55.628173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:17:55.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:55.628750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:17:55.628750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:55.628750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:17:57.816167 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:17:57.816567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:17:57.816567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:17:57.831987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 27 16:17:57.832798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Number of inconsistencies found at create/update: networks=7, security_groups=1, routers=4, router_ports=4 {{(pid=92285) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 27 16:17:57.833102 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Number of inconsistencies found at delete: router_ports=19 {{(pid=92285) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 27 16:17:57.833251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e2510e0f-28d1-484d-8605-6e6fbc1e01e2 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:57.872452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e2510e0f-28d1-484d-8605-6e6fbc1e01e2, resource={'id': 'e2510e0f-28d1-484d-8605-6e6fbc1e01e2', 'name': 'private', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2524c773-2910-41f1-b8dd-03d9ac809fd5', '71e0c14f-375e-43c9-bf49-7597bd311ee9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T15:57:42Z', 'updated_at': '2026-05-27T15:57:44Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 8519}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.872688 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e2510e0f-28d1-484d-8605-6e6fbc1e01e2, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.873001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-96ddfe89-0ab3-4b19-b3ee-13a966e7d5c2, external_ids_update=None, columns={'tag': 8519}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.873108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.873800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:57.873800 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:57.873800 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:57.873800 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:57.873800 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:57.873800 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:57.873800 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.873800 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource e2510e0f-28d1-484d-8605-6e6fbc1e01e2 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:57.875277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:57.875488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:57.875722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:57.875722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.875722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:57.875722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1a1659c0-4943-4985-b3f1-75f0c26af59f (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:57.904195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, resource={'id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'name': 'public', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['156facd3-ecc0-4e78-8b24-b1c2e5631a78', '3c538124-e9fa-4f89-905c-f13de628c91d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-27T15:57:46Z', 'updated_at': '2026-05-27T15:57:53Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.904483 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.904862 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1d74a028-7065-4269-8c77-c99e0ea79147, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.905096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.905777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:57.905777 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:57.905777 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:57.905777 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:57.905777 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:57.905777 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:57.905777 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.905777 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 1a1659c0-4943-4985-b3f1-75f0c26af59f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:57.906724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:57.906926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:57.907148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:57.907148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.907148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:57.907148 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 28588915-92a2-4ece-bd6f-c59cf1c236ce (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:57.929372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28588915-92a2-4ece-bd6f-c59cf1c236ce, resource={'id': '28588915-92a2-4ece-bd6f-c59cf1c236ce', 'name': 'shared', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['73ef14d1-56af-457d-a4cc-cd982d0f7719'], 'standard_attr_id': 26, 'shared': True, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T15:58:53Z', 'updated_at': '2026-05-27T15:58:54Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 65356}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.929669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28588915-92a2-4ece-bd6f-c59cf1c236ce, col_values=(('external_ids', {'neutron:network_name': 'shared', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.930021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-854ee673-4c8a-4fe6-bb8f-35261288982d, external_ids_update=None, columns={'tag': 65356}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.930144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.930823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:57.930823 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:57.930823 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:57.930823 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:57.930823 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:57.930823 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:57.930823 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.930823 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 28588915-92a2-4ece-bd6f-c59cf1c236ce (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:57.931909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:57.932219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:57.932557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:57.932557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.932557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:57.932557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 786909f4-f05b-48cc-be9a-98a0de55d4d1 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:57.955698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-786909f4-f05b-48cc-be9a-98a0de55d4d1, resource={'id': '786909f4-f05b-48cc-be9a-98a0de55d4d1', 'name': 'tempest-TestEncryptedCinderVolumes-1796942463-network', 'tenant_id': '3891fbf9ca22459db7309e558c3235fd', 'project_id': '3891fbf9ca22459db7309e558c3235fd', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2f2aee5c-9d1e-4edf-a2b8-53c196b95ec9'], 'standard_attr_id': 1533, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:13:53Z', 'updated_at': '2026-05-27T16:13:54Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 28247}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.955995 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-786909f4-f05b-48cc-be9a-98a0de55d4d1, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestEncryptedCinderVolumes-1796942463-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.956326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7d479754-9403-47b7-8a56-5e4e7b37a93a, external_ids_update=None, columns={'tag': 28247}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.956451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.957165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:57.957165 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:57.957165 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:57.957165 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:57.957165 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:57.957165 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:57.957165 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.957165 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 786909f4-f05b-48cc-be9a-98a0de55d4d1 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:57.957986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:57.958184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:57.958457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:57.958457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.958457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:57.958457 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource cdf92c13-0320-4430-8ccc-b66465f08b7f (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:57.981132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-cdf92c13-0320-4430-8ccc-b66465f08b7f, resource={'id': 'cdf92c13-0320-4430-8ccc-b66465f08b7f', 'name': 'tempest-TestEncryptedCinderVolumes-1334460186-network', 'tenant_id': 'b7b4bc0b0dba4564bbb4787fc0140873', 'project_id': 'b7b4bc0b0dba4564bbb4787fc0140873', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8c056a73-612f-491f-8553-6612a508b71c'], 'standard_attr_id': 1557, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:14:01Z', 'updated_at': '2026-05-27T16:14:01Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 38592}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.981427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-cdf92c13-0320-4430-8ccc-b66465f08b7f, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestEncryptedCinderVolumes-1334460186-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.981745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-11a89f75-d469-48c0-aba3-7e1130aeafa3, external_ids_update=None, columns={'tag': 38592}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:57.981883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.982554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:57.982554 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:57.982554 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:57.982554 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:57.982554 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:57.982554 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:57.982554 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.982554 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource cdf92c13-0320-4430-8ccc-b66465f08b7f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:57.983418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:57.983643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:57.983895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:57.983895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:57.983895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:57.983895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource ec56e636-ffbb-4d63-87d1-b6b1dc7be102 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.007216 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-ec56e636-ffbb-4d63-87d1-b6b1dc7be102, resource={'id': 'ec56e636-ffbb-4d63-87d1-b6b1dc7be102', 'name': 'tempest-TestVolumeBootPattern-1623358333-network', 'tenant_id': '1d9d90a978294926918bf1013e804d9b', 'project_id': '1d9d90a978294926918bf1013e804d9b', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['821ea963-808e-4b5e-a790-198868ae4a46'], 'standard_attr_id': 1721, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:16:10Z', 'updated_at': '2026-05-27T16:16:11Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 35046}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.007475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-ec56e636-ffbb-4d63-87d1-b6b1dc7be102, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestVolumeBootPattern-1623358333-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.007806 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3a7bbffa-67a2-49bc-a12e-56c59ede0c97, external_ids_update=None, columns={'tag': 35046}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.007911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.008550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:58.008550 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.008550 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:58.008550 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:58.008550 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.008550 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:58.008550 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.008550 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource ec56e636-ffbb-4d63-87d1-b6b1dc7be102 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.009385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:58.009651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.009874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:58.009874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.009874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:58.009874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 60e7b876-c3cc-4dbc-8d09-ec2fe5b5917a (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.033850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-60e7b876-c3cc-4dbc-8d09-ec2fe5b5917a, resource={'id': '60e7b876-c3cc-4dbc-8d09-ec2fe5b5917a', 'name': 'tempest-TestVolumeBootPattern-1412527984-network', 'tenant_id': '514502f614654a7ea82df88f84bee31a', 'project_id': '514502f614654a7ea82df88f84bee31a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['a5367ee9-c0cf-472f-9ab0-6e77a12b5514'], 'standard_attr_id': 1744, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:16:17Z', 'updated_at': '2026-05-27T16:16:17Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 63684}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.034165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-60e7b876-c3cc-4dbc-8d09-ec2fe5b5917a, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestVolumeBootPattern-1412527984-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.034477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b8382ca-0035-47c2-9348-56afe07334c2, external_ids_update=None, columns={'tag': 63684}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.034609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.035311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:58.035311 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.035311 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:58.035311 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:58.035311 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.035311 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:58.035311 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.035311 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 60e7b876-c3cc-4dbc-8d09-ec2fe5b5917a (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.036366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:58.036669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.036902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:58.036902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.036902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:58.036902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1731e623-a71e-4f27-aad5-bd00b6336164 (type: security_groups) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.063388 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Successfully bumped revision number for resource 1731e623-a71e-4f27-aad5-bd00b6336164 (type: security_groups) to 7
May 27 16:17:58.063533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1d4d4360-9982-47ff-a481-af1082fb5fb5 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.123152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, resource={'id': '1d4d4360-9982-47ff-a481-af1082fb5fb5', 'name': 'tempest-TestEncryptedCinderVolumes-1796942463-router', 'project_id': '3891fbf9ca22459db7309e558c3235fd', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.212'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::3c4'}], 'enable_snat': True}, 'gw_port_id': '41077fdf-a196-47b3-8ac4-e96813086e9a', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.212', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::3c4', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:13:54Z', 'updated_at': '2026-05-27T16:13:58Z', 'revision_number': 3, 'tenant_id': '3891fbf9ca22459db7309e558c3235fd'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.123415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, columns={'external_ids': {'neutron:router_name': 'tempest-TestEncryptedCinderVolumes-1796942463-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.123639 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "41077fdf-a196-47b3-8ac4-e96813086e9a"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.123794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "41077fdf-a196-47b3-8ac4-e96813086e9a"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.123911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.124536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:58.124536 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.124536 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:58.124536 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:58.124536 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.124536 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:58.124536 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.124536 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:58.124714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 1d4d4360-9982-47ff-a481-af1082fb5fb5. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 1d4d4360-9982-47ff-a481-af1082fb5fb5 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:17:58.125736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:58.125941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:58.126160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:58.126160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:58.126160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.126160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:58.126160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:58.126160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.126160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:58.126160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.126160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:58.126160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 41a9a807-84f6-48fc-abb7-de78f5766371 (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.192597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, resource={'id': '41a9a807-84f6-48fc-abb7-de78f5766371', 'name': 'tempest-TestEncryptedCinderVolumes-1334460186-router', 'project_id': 'b7b4bc0b0dba4564bbb4787fc0140873', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.143'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::2e'}], 'enable_snat': True}, 'gw_port_id': '7d877385-7424-4d92-9669-65a6b38e2aa9', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.143', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::2e', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:14:02Z', 'updated_at': '2026-05-27T16:14:05Z', 'revision_number': 3, 'tenant_id': 'b7b4bc0b0dba4564bbb4787fc0140873'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.192877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, columns={'external_ids': {'neutron:router_name': 'tempest-TestEncryptedCinderVolumes-1334460186-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.193101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "7d877385-7424-4d92-9669-65a6b38e2aa9"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.193259 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "7d877385-7424-4d92-9669-65a6b38e2aa9"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.193380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.194087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:58.194087 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.194087 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:58.194087 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:58.194087 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.194087 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:58.194087 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.194087 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:58.194465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router 41a9a807-84f6-48fc-abb7-de78f5766371. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 41a9a807-84f6-48fc-abb7-de78f5766371 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:17:58.195519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:58.195756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:58.196034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:58.196034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:58.196034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.196034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:58.196034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:58.196034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.196034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:58.196034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.196034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:58.196034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource dbce49bb-17ac-40db-91aa-b0d13e58bb7b (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.256802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, resource={'id': 'dbce49bb-17ac-40db-91aa-b0d13e58bb7b', 'name': 'tempest-TestVolumeBootPattern-1623358333-router', 'project_id': '1d9d90a978294926918bf1013e804d9b', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.109'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::bd'}], 'enable_snat': True}, 'gw_port_id': '513a2a7a-94d2-4cba-a15e-1f67ba36637d', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.109', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::bd', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:16:11Z', 'updated_at': '2026-05-27T16:16:14Z', 'revision_number': 3, 'tenant_id': '1d9d90a978294926918bf1013e804d9b'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.257088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, columns={'external_ids': {'neutron:router_name': 'tempest-TestVolumeBootPattern-1623358333-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.257283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "513a2a7a-94d2-4cba-a15e-1f67ba36637d"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.257409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "513a2a7a-94d2-4cba-a15e-1f67ba36637d"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.257504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.258113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:58.258113 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.258113 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:58.258113 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:58.258113 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.258113 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:58.258113 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.258113 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:58.258272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router dbce49bb-17ac-40db-91aa-b0d13e58bb7b. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource dbce49bb-17ac-40db-91aa-b0d13e58bb7b (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:17:58.259285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:58.259496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:58.259783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:58.259783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:58.259783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.259783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:58.259783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:58.259783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.259783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:58.259783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.259783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:58.259783 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource cb0982dc-f84e-421b-ae33-411ef007397f (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.318759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, resource={'id': 'cb0982dc-f84e-421b-ae33-411ef007397f', 'name': 'tempest-TestVolumeBootPattern-1412527984-router', 'project_id': '514502f614654a7ea82df88f84bee31a', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.232'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::1de'}], 'enable_snat': True}, 'gw_port_id': '03060c4d-30ec-4bc1-96d0-168d95e31a89', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.232', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::1de', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:16:18Z', 'updated_at': '2026-05-27T16:16:20Z', 'revision_number': 3, 'tenant_id': '514502f614654a7ea82df88f84bee31a'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.319002 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, columns={'external_ids': {'neutron:router_name': 'tempest-TestVolumeBootPattern-1412527984-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.319165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "03060c4d-30ec-4bc1-96d0-168d95e31a89"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.319278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "03060c4d-30ec-4bc1-96d0-168d95e31a89"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.319382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.319983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:58.319983 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.319983 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:58.319983 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:58.319983 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.319983 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:58.319983 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.319983 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:58.320170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router cb0982dc-f84e-421b-ae33-411ef007397f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource cb0982dc-f84e-421b-ae33-411ef007397f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:17:58.321265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:58.321724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:58.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:58.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:58.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:58.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:58.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:58.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:58.322067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 41077fdf-a196-47b3-8ac4-e96813086e9a (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.409966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-41077fdf-a196-47b3-8ac4-e96813086e9a, resource={'id': '41077fdf-a196-47b3-8ac4-e96813086e9a', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:60:11:03', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1d4d4360-9982-47ff-a481-af1082fb5fb5', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1540, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.212'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::3c4'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:13:55Z', 'updated_at': '2026-05-27T16:13:58Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.410197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-41077fdf-a196-47b3-8ac4-e96813086e9a, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::3c4/64', '172.24.5.212/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=41077fdf-a196-47b3-8ac4-e96813086e9a, lrouter_port=lrp-41077fdf-a196-47b3-8ac4-e96813086e9a, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.410660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.411167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:58.411167 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.411167 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:58.411167 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:58.411167 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.411167 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:58.411167 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.411167 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 41077fdf-a196-47b3-8ac4-e96813086e9a (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.411926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:58.412147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.412424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:58.412424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.412424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:58.412424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 7d877385-7424-4d92-9669-65a6b38e2aa9 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.495434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-7d877385-7424-4d92-9669-65a6b38e2aa9, resource={'id': '7d877385-7424-4d92-9669-65a6b38e2aa9', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:e2:91:9b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '41a9a807-84f6-48fc-abb7-de78f5766371', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1562, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.143'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::2e'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:14:02Z', 'updated_at': '2026-05-27T16:14:07Z', 'revision_number': 7, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.495609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-7d877385-7424-4d92-9669-65a6b38e2aa9, columns={'external_ids': {'neutron:revision_number': '7', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-41a9a807-84f6-48fc-abb7-de78f5766371'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.143/24', '2001:db8::2e/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.495833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=7d877385-7424-4d92-9669-65a6b38e2aa9, lrouter_port=lrp-7d877385-7424-4d92-9669-65a6b38e2aa9, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.496003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.496444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:58.496444 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.496444 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:58.496444 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:58.496444 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.496444 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:58.496444 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.496444 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 7d877385-7424-4d92-9669-65a6b38e2aa9 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.497221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:58.497707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.498034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:58.498034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.498034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:58.498034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 513a2a7a-94d2-4cba-a15e-1f67ba36637d (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.579801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-513a2a7a-94d2-4cba-a15e-1f67ba36637d, resource={'id': '513a2a7a-94d2-4cba-a15e-1f67ba36637d', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:d9:3c:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dbce49bb-17ac-40db-91aa-b0d13e58bb7b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1726, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.109'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::bd'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:16:12Z', 'updated_at': '2026-05-27T16:16:15Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.580029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-513a2a7a-94d2-4cba-a15e-1f67ba36637d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.109/24', '2001:db8::bd/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.580283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=513a2a7a-94d2-4cba-a15e-1f67ba36637d, lrouter_port=lrp-513a2a7a-94d2-4cba-a15e-1f67ba36637d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.580517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.581017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:58.581017 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.581017 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:58.581017 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:58.581017 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.581017 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:58.581017 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.581017 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 513a2a7a-94d2-4cba-a15e-1f67ba36637d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.581699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:58.581926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.582161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:58.582161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.582161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:58.582161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 03060c4d-30ec-4bc1-96d0-168d95e31a89 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:17:58.680029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-03060c4d-30ec-4bc1-96d0-168d95e31a89, resource={'id': '03060c4d-30ec-4bc1-96d0-168d95e31a89', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:c6:36:b9', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cb0982dc-f84e-421b-ae33-411ef007397f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1749, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.232'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::1de'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:16:18Z', 'updated_at': '2026-05-27T16:16:22Z', 'revision_number': 7, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.680214 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-03060c4d-30ec-4bc1-96d0-168d95e31a89, columns={'external_ids': {'neutron:revision_number': '7', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-cb0982dc-f84e-421b-ae33-411ef007397f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::1de/64', '172.24.5.232/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.680415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=03060c4d-30ec-4bc1-96d0-168d95e31a89, lrouter_port=lrp-03060c4d-30ec-4bc1-96d0-168d95e31a89, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:17:58.680608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.681277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:17:58.681277 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.681277 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:17:58.681277 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:17:58.681277 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.681277 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:17:58.681277 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.681277 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 03060c4d-30ec-4bc1-96d0-168d95e31a89 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.681977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:17:58.682200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:17:58.682472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:17:58.682472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:17:58.682472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:17:58.682472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 60e4e806-5ed0-4260-b411-bdb6e7afb86a (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.682472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(60e4e806-5ed0-4260-b411-bdb6e7afb86a) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.687356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e6dec0ad-e7e9-429d-b499-4aba52ec4762 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.687711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(e6dec0ad-e7e9-429d-b499-4aba52ec4762) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.691770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource c7e4f10e-bf5f-41a2-be16-a1cbeca114f5 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.692009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(c7e4f10e-bf5f-41a2-be16-a1cbeca114f5) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.697122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource c3eab4b6-b2f7-455d-a6ff-eee75a2f479f (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.697444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(c3eab4b6-b2f7-455d-a6ff-eee75a2f479f) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.701661 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource a739750f-d85d-4117-83ac-17a265e2d010 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.701887 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(a739750f-d85d-4117-83ac-17a265e2d010) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.708426 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource a3f5d312-05e6-4ce9-a561-14f2fde17de9 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.708655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(a3f5d312-05e6-4ce9-a561-14f2fde17de9) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.712925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 907418ec-c571-42e5-ac21-bb421730feb4 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.713200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(907418ec-c571-42e5-ac21-bb421730feb4) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.717661 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 733d57d2-3926-45a6-8788-f020194dc56c (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.717890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(733d57d2-3926-45a6-8788-f020194dc56c) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.721821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 71eed332-cd4a-4cdb-8d2e-b6135a27facc (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.722037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(71eed332-cd4a-4cdb-8d2e-b6135a27facc) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.725754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 610ecf1c-5a5c-4559-b303-28748c8c094d (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.725979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(610ecf1c-5a5c-4559-b303-28748c8c094d) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.729691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 03f8d119-502d-4acd-b55d-9da1ba4109ba (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.729907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(03f8d119-502d-4acd-b55d-9da1ba4109ba) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.735184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 5e3f0db1-92c1-468d-acca-da97845bcb97 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.735408 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(5e3f0db1-92c1-468d-acca-da97845bcb97) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.739895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 5600263a-390a-4b19-be3c-a80c416965ff (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.740135 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(5600263a-390a-4b19-be3c-a80c416965ff) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.752586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 38733f67-cd59-4bb8-acef-1cffd44d0e32 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.753788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(38733f67-cd59-4bb8-acef-1cffd44d0e32) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.768328 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 3448491d-a9ef-42fd-ae45-d63ea6439591 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.768580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(3448491d-a9ef-42fd-ae45-d63ea6439591) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.780767 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 2ecb6a02-e73d-4a32-91d8-2a03271bcd04 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.781865 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(2ecb6a02-e73d-4a32-91d8-2a03271bcd04) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.797244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 2e3c6a7f-ad57-484d-9577-0a5aa6082148 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.798123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(2e3c6a7f-ad57-484d-9577-0a5aa6082148) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.804940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1c60c96b-c574-4c02-aa9e-c92275d798c8 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.805162 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(1c60c96b-c574-4c02-aa9e-c92275d798c8) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.810385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1686d8b9-efb5-45d6-b1d8-96e0c0968127 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:17:58.810608 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(1686d8b9-efb5-45d6-b1d8-96e0c0968127) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:17:58.815037 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_for_inconsistencies finished in 0.998 seconds
May 27 16:18:00.342389 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:00.342799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:00.342799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:00.343265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:00.369767 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.369955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.370455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:00.370455 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.370455 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:00.370455 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:00.370455 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.370455 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:00.370455 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.370455 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.371070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.371456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:00.371817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:00.372239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.372824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:00.372824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:00.372824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.372824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:00.372824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.372824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:00.587065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:00.587316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:00.587404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:00.588228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.588430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.588474 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.588578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.588685 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.588794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.588913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.589032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.589178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.589844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:00.589844 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.589844 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:00.589844 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:00.589844 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.589844 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:00.589844 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.589844 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.590586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.590916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:00.591162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.591162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:00.591286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:00.591702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.592075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:00.592075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.592075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:00.599441 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:00.599668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:00.599778 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:00.599929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:00.600322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.600496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.600996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:00.600996 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.600996 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:00.600996 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:00.600996 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.600996 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:00.600996 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.600996 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:00.601596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:00.601945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.601945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:00.601945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.601945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:00.601945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:00.601945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.601945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:00.601945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.601945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.602373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:00.602639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:00.602939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.602939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:00.602939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.602939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:00.603369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.603724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:00.604062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.604062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:00.604062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:00.604062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.604062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:00.604062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.604062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:00.611220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:00.611510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:00.611633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:00.611780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:00.612104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.612286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.612717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:00.612717 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.612717 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:00.612717 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:00.612717 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.612717 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:00.612717 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.612717 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:00.613424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:00.613894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.613894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:00.613894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.613894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:00.613894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:00.613894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.613894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:00.613894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.613894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.614289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:00.614520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:00.614802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.614802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:00.614802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.614802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.615243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:00.615673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:00.616030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.616030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:00.616030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.616030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:00.624486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:00.624747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:00.624861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:00.625043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:00.625383 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.625633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:00.625900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.626427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:00.626427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.626427 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:00.626427 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:00.626427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.626427 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:00.626427 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.626427 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.627210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.627472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:00.627761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.627761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:00.628161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:00.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:00.628800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:00.628800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:00.628800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:05.342840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:05.343460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:05.343713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:05.349950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:05.385725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.385951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.386487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:05.386487 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.386487 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:05.386487 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:05.386487 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.386487 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:05.386487 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.386487 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.387171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.387394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:05.387588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.387588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:05.387870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:05.388180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.388443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:05.388443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.388443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:05.587553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:05.589480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:05.589596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:05.590636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.590857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.590981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.591080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.591214 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.591321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.591424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.591520 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.591631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.592091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:05.592091 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.592091 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:05.592091 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:05.592091 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.592091 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:05.592091 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.592091 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.592981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.593300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:05.593636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.593636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:05.596468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:05.596892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.597335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:05.597335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.597335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:05.599705 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:05.599941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:05.600070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:05.600211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:05.600522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.600686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.601953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:05.601953 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.601953 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:05.601953 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:05.601953 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.601953 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:05.601953 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.601953 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:05.602634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:05.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:05.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:05.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:05.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:05.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.603531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:05.603868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:05.604203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.604203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:05.604203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.604203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:05.609261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.609674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:05.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:05.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:05.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:05.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:05.611578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:05.611853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:05.611968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:05.612121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:05.612462 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.612645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.613070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:05.613070 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.613070 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:05.613070 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:05.613070 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.613070 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:05.613070 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.613070 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:05.613896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:05.614256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.614256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:05.614256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.614256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:05.614256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:05.614256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.614256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:05.614256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.614256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.614833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:05.615126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:05.615458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.615458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:05.615458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.615458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.616006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:05.616426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:05.616883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.616883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:05.616883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.616883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:05.624917 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:05.626918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:05.627037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:05.627181 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:05.627533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.627754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:05.627953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.628392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:05.628392 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.628392 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:05.628392 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:05.628392 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.628392 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:05.628392 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.628392 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.629078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.629372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:05.629716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.629716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:05.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:05.630154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:05.630579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:05.630579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:05.630579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:10.343361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:10.343929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:10.343929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:10.344727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:10.382781 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.383634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:10.383634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.383634 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:10.383634 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:10.383634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.383634 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:10.383634 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.383634 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.384677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.384955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:10.385197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.385197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:10.385750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:10.386076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.386601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:10.386601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.386601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:10.589587 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:10.590128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:10.590192 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:10.591156 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.591436 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.591436 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.591616 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.591757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.591887 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.592014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.592143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.592900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.592953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:10.592953 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.592953 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:10.592953 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:10.592953 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.592953 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:10.592953 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.592953 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.594179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.594922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:10.595139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.595139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:10.595486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:10.596905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.597322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:10.597322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.597322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:10.600010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:10.601695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:10.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:10.601886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:10.602281 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.602469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:10.603011 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.603011 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:10.603011 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:10.603011 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.603011 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:10.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.603011 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:10.603608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:10.603890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.603890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:10.603890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.603890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:10.603890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:10.603890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.603890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:10.603890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.603890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.604191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:10.604393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:10.604669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.604669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:10.604669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.604669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:10.605089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.606328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:10.607143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.607143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:10.607143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:10.607143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.607143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:10.607143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.607143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:10.612235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:10.612235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:10.612235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:10.616140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:10.616776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:10.617321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:10.617813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:10.618093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.618344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.620096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:10.628656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:10.628920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:10.629040 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:10.629219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:10.629558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.629825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:10.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:10.630739 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.630739 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:10.630739 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:10.630739 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.630739 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:10.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.631458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.631722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:10.631921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.631921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:10.632455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:10.632744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:10.633036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:10.633036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:10.633036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:15.343774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:15.344293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:15.344458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:15.345501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:15.381203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.381424 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.382003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:15.382003 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.382003 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:15.382003 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:15.382003 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.382003 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:15.382003 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.382003 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.382991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:15.383190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.383190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:15.383669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:15.384112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.384408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:15.384408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.384408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:15.590056 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:15.590450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:15.590522 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:15.591418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.591599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.591743 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.591867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.591999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.592103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.592211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.592333 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.592444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.592912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:15.592912 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.592912 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:15.592912 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:15.592912 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.592912 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:15.592912 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.592912 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.593606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.593881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:15.594088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.594088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:15.594373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:15.594773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.595051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:15.595051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.595051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:15.601851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:15.602070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:15.602151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:15.602246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:15.602578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.602775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.603230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:15.603230 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.603230 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:15.603230 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:15.603230 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.603230 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:15.603230 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.603230 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:15.603776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:15.604020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.604020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:15.604020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.604020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:15.604020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:15.604020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.604020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:15.604020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.604020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.604388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:15.604680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:15.604920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.604920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:15.604920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.604920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:15.604986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.605218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:15.605477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.605477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:15.605477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:15.605477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.605477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:15.605477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.605477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:15.615392 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:15.615609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:15.615725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:15.615824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:15.616144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.616365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.616945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:15.616945 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.616945 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:15.616945 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:15.616945 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.616945 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:15.616945 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.616945 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:15.617384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:15.617683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.617683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:15.617683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.617683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:15.617683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:15.617683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.617683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:15.617683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.617683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.618116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:15.618396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:15.618642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:15.618862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.619137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:15.629054 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:15.629287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:15.629381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:15.629532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:15.629878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.630229 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:15.630271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.630768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:15.630768 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.630768 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:15.630768 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:15.630768 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.630768 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:15.630768 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.630768 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.632869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:15.633336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:15.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:15.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:15.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:15.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:15.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:15.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:20.344232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:20.344929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:20.344929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:20.345266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:20.376006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.376238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.376808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:20.376808 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.376808 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:20.376808 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:20.376808 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.376808 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:20.376808 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.376808 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.377532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.377798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:20.377990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.377990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:20.378274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:20.378566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.378918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:20.378918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.378918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:20.590563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:20.590965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:20.591080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:20.592066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.592402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.592686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.592842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.592996 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.593148 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.593305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.593443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.593640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.594351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:20.594351 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.594351 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:20.594351 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:20.594351 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.594351 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:20.594351 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.594351 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.595074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.595355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:20.595551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.595551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:20.595868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:20.596163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.596500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:20.596500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.596500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:20.602272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:20.602559 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:20.602699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:20.602834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:20.603300 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.603515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.604068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:20.604068 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.604068 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:20.604068 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:20.604068 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.604068 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:20.604068 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.604068 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:20.604609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:20.604883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.604883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:20.604883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.604883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:20.604883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:20.604883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.604883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:20.604883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.604883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.605178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:20.605386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:20.605587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.605587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:20.605587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.605587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:20.606610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.606900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:20.607207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.607207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:20.607207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:20.607207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.607207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:20.607207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.607207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:20.615774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:20.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:20.616133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:20.616263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:20.616665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.616871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.617320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:20.617320 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.617320 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:20.617320 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:20.617320 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.617320 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:20.617320 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.617320 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:20.617894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:20.618166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.618166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:20.618166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.618166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:20.618166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:20.618166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.618166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:20.618166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.618166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.618472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:20.618683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:20.618897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.618897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:20.618897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.618897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.619166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:20.619454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:20.619765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.619765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:20.619765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.619765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:20.629427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:20.629728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:20.629854 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:20.630018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:20.630406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.630634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:20.630858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.631288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:20.631288 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.631288 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:20.631288 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:20.631288 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.631288 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:20.631288 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.631288 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.631894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.632103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:20.632303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.632303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:20.632728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:20.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:20.633287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:20.633287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:20.633287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:25.344657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:25.345248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:25.345248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:25.346106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:25.377827 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.378123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.378766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:25.378766 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.378766 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:25.378766 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:25.378766 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.378766 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:25.378766 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.378766 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.379475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.379738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:25.379933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.379933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:25.380238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:25.380530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.380889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:25.380889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.380889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:25.591410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:25.592971 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:25.593528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:25.596402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.596679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.596894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.597052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.597219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.597392 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.597533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.597706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.597851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.598446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:25.598446 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.598446 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:25.598446 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:25.598446 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.598446 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:25.598446 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.598446 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.601428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.602407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:25.602845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:25.603262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.603768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:25.603768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:25.603768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.603768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:25.603768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.603768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:25.603768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:25.603768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:25.603768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:25.603768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:25.604073 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.604228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.604677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:25.604677 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.604677 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:25.604677 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:25.604677 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.604677 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:25.604677 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.604677 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:25.605512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:25.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:25.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:25.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:25.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:25.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.606092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:25.606345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:25.606678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.606678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:25.606678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.606678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:25.606818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.607197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:25.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:25.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:25.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:25.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.607609 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:25.616075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:25.616233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:25.616331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:25.616439 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:25.616854 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.617061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.617518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:25.617518 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.617518 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:25.617518 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:25.617518 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.617518 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:25.617518 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.617518 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:25.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:25.618659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.618659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:25.618659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.618659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:25.618659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:25.618659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.618659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:25.618659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.618659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.618902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:25.619280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:25.619645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:25.620037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.620544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:25.630589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:25.630841 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:25.630935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:25.631099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:25.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.631601 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:25.631846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.632269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:25.632269 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.632269 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:25.632269 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:25.632269 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.632269 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:25.632269 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.632269 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.632941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.633358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:25.633575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.633575 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:25.633615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:25.633914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:25.634329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:25.634329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:25.634329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:30.345101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:30.345583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:30.345695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:30.346298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:30.387984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.388233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.388826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:30.388826 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.388826 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:30.388826 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:30.388826 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.388826 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:30.388826 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.388826 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.389552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.390053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:30.390406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:30.390814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.391328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:30.391328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:30.391328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.391328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:30.391328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.391328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:30.592114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:30.592577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:30.592577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:30.593975 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.593975 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.594285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.594347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.594486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.594594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.594739 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.594854 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.594964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.595536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:30.595536 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.595536 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:30.595536 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:30.595536 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.595536 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:30.595536 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.595536 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.596762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.597055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:30.598048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:30.598997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.599439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:30.599439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:30.599439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.599439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:30.599439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.599439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:30.603676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:30.603879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:30.604003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:30.604536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:30.604536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.605221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.605330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:30.605330 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.605330 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:30.605330 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:30.605330 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.605330 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:30.605330 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.605330 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:30.606140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:30.606870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.606870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:30.606870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.606870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:30.606870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:30.606870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.606870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:30.606870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.606870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.607064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:30.607914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:30.608520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.609783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.610209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:30.616581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:30.617178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:30.617178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:30.617178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:30.617577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.617993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.618398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:30.618398 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.618398 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:30.618398 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:30.618398 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.618398 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:30.618398 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.618398 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:30.619523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:30.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:30.620982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:30.622213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:30.623425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.624693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:30.631113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:30.631473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:30.631963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:30.631963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:30.632051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.632342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:30.632495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.632914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:30.632914 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.632914 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:30.632914 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:30.632914 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.632914 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:30.632914 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.632914 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.634065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:30.634294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.634294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:30.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:30.635571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:30.636908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:30.636908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:30.636908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:35.345491 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:35.345891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:35.345996 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:35.346641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:35.375289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.375563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.376024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:35.376024 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.376024 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:35.376024 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:35.376024 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.376024 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:35.376024 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.376024 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.376636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.376900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:35.377104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.377104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:35.377330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:35.377590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.377941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:35.377941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.377941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:35.592500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:35.592908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:35.593009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:35.593820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.593963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.594075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.594200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.594383 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.594490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.594612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.594745 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.594851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.595254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:35.595254 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.595254 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:35.595254 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:35.595254 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.595254 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:35.595254 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.595254 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.595896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.596109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:35.596297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.596297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:35.596672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:35.596981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.597296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:35.597296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.597296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:35.603936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:35.604208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:35.604298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:35.604391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:35.604665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.604822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.605189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:35.605189 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.605189 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:35.605189 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:35.605189 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.605189 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:35.605189 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.605189 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:35.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:35.605965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.605965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:35.605965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.605965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:35.605965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:35.605965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.605965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:35.605965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.605965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.606298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:35.606498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:35.606728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.606728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:35.606728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.606728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:35.606991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.607232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:35.607489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.607489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:35.607489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:35.607489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.607489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:35.607489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.607489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:35.616850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:35.617228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:35.617356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:35.617466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:35.617800 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.618006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.618402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:35.618402 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.618402 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:35.618402 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:35.618402 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.618402 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:35.618402 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.618402 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:35.618911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:35.619360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.619360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:35.619360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.619360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:35.619360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:35.619360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.619360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:35.619360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.619360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.619672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:35.620006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:35.620432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:35.620894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.621450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:35.631512 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:35.631764 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:35.631837 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:35.631980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:35.632195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.632354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:35.632747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.632885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:35.632885 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.632885 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:35.632885 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:35.632885 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.632885 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:35.632885 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.632885 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.633841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:35.634222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:35.634719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:35.635343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:35.635343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:35.635343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:35.635343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:35.635343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:35.635343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:40.346009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:40.346654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:40.346654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:40.347467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:40.379981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.380215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.380762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:40.380762 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.380762 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:40.380762 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:40.380762 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.380762 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:40.380762 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.380762 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.381423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.381693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:40.381912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.381912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:40.382094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:40.382463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.382817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:40.382817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.382817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:40.593048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:40.593467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:40.593613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:40.594484 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.594707 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-41a9a807-84f6-48fc-abb7-de78f5766371, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.594845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.594980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.595123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.595253 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.595388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=6): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.595526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=7): DelStaticRoutesCommand(_result=None, lrouter=neutron-1d4d4360-9982-47ff-a481-af1082fb5fb5, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.595673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.596246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:40.596246 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.596246 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:40.596246 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:40.596246 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.596246 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:40.596246 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.596246 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.597139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.597417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:40.597711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.597711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:40.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:40.598562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.598957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:40.598957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.598957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:40.604427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:40.604710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:40.604849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:40.604981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:40.605319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.605500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.605904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:40.605904 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.605904 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:40.605904 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:40.605904 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.605904 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:40.605904 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.605904 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:40.606437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:40.606896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.606896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:40.606896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.606896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:40.606896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:40.606896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.606896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:40.606896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.606896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:40.607504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:40.607926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.608336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.608878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:40.617413 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:40.618025 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:40.618145 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:40.618430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:40.618673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.618879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.619361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:40.619361 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.619361 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:40.619361 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:40.619361 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.619361 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:40.619361 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.619361 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:40.619977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:40.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:40.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:40.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:40.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:40.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.620717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:40.621193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:40.621702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:40.621973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.622384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:40.631923 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:40.632242 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:40.632333 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:40.632471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:40.632828 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:40.633281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.633812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:40.633812 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.633812 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:40.633812 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:40.633812 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.633812 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:40.633812 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.633812 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.634449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.634748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:40.634997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.634997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:40.635475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:40.635816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:40.636114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:40.636114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:40.636114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:45.346463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:45.347001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:45.347001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:45.347581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:45.375822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.375976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.376403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:45.376403 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.376403 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:45.376403 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:45.376403 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.376403 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:45.376403 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.376403 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.376989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.377216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:45.377427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.377427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:45.377685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:45.377944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.378224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:45.378224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.378224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:45.593643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:45.593908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:45.594030 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:45.594813 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.594987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.595133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.595278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.595430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.595568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.595721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.596184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:45.596184 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.596184 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:45.596184 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:45.596184 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.596184 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:45.596184 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.596184 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.596956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.597356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:45.597813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.597813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:45.598130 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:45.598583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:45.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.599147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:45.604831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:45.605106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:45.605223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:45.605376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:45.605689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.605845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.606252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:45.606252 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.606252 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:45.606252 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:45.606252 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.606252 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:45.606252 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.606252 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:45.606911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:45.607316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.607316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:45.607316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.607316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:45.607316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:45.607316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.607316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:45.607316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.607316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.608135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:45.608496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:45.608940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.608940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:45.608940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.608940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:45.609052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.609309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:45.609657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.609657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:45.609657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:45.609657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.609657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:45.609657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.609657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:45.618101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:45.618636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:45.618636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:45.618636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:45.618885 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.619097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.619686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:45.619686 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.619686 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:45.619686 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:45.619686 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.619686 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:45.619686 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.619686 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:45.620460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:45.620917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:45.621254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.621590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.621978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:45.622303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:45.622640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.622640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:45.622640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.622640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:45.632252 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:45.632520 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:45.632695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:45.632882 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:45.633143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.633351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:45.633537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.633927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:45.633927 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.633927 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:45.633927 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:45.633927 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.633927 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:45.633927 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.633927 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.634612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.634975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:45.635342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.635342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:45.635633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:45.635941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:45.636236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:45.636236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:45.636236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:50.346897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:50.347308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:50.347308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:50.347860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:50.378899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.379243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.380584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:50.380584 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.380584 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:50.380584 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:50.380584 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.380584 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:50.380584 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.380584 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.381650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.381866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:50.382139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.382139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:50.382770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:50.383033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.383363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:50.383363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.383363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:50.594077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:50.595038 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:50.595038 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:50.595209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.595324 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.595429 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.595529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.595633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.595733 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.595853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:50.597519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:50.598349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:50.598709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.599022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:50.599384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:50.599384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.599384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:50.599384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.599384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:50.599384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:50.599384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.599384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:50.599384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.599384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:50.605212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:50.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:50.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:50.605884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:50.606339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.606572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.607435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:50.607435 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.607435 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:50.607435 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:50.607435 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.607435 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:50.607435 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.607435 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:50.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:50.608024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.608024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:50.608024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.608024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:50.608024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:50.608024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.608024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:50.608024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.608024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.608313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:50.608669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:50.608933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.608933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:50.608933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.608933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:50.609062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.609352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:50.609720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.609720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:50.609720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:50.609720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.609720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:50.609720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.609720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:50.618513 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:50.619298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:50.619298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:50.619298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:50.620197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.620197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.620974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:50.620974 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.620974 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:50.620974 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:50.620974 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.620974 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:50.620974 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.620974 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:50.621235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:50.621655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:50.622037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:50.622416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:50.622757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.623254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:50.632671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:50.633272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:50.633272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:50.633346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:50.633713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.633958 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:50.634196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.634744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:50.634744 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.634744 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:50.634744 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:50.634744 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.634744 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:50.634744 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.634744 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.635487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.635756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:50.635982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.635982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:50.636227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:50.636565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:50.636898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:50.636898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:50.636898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:55.348492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:55.348911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:55.348911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:55.349382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:55.353161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=858,neutron:UPDATE=1,neutron:DELETE=19 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:18:55.376362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.376571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.377153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:55.377153 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.377153 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:55.377153 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:55.377153 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.377153 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:55.377153 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.377153 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.377765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.378008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:55.378215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.378215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:55.378557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:55.379235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.379603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:55.379603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.379603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:55.594602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:55.594899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:55.594988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:55.595722 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.595857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.595959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.596104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.596203 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.596295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.596386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.596802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:55.596802 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.596802 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:55.596802 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:55.596802 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.596802 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:55.596802 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.596802 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.597384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.597599 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:55.597820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.597820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:55.598085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:55.598363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.598661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:55.598661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.598661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:55.605520 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:55.605678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:55.605756 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:55.605840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:18:55.606075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.606245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.606535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:55.606535 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.606535 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:55.606535 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:55.606535 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.606535 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:55.606535 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.606535 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:55.607102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:55.607399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.607399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:55.607399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.607399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:55.607399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:55.607399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.607399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:55.607399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.607399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.607619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:55.607853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:55.608094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.608094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:55.608094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.608094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:55.608282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.608546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:55.608854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.608854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:55.608854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:55.608854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.608854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:55.608854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.608854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:55.618864 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:55.619069 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:55.619134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:55.619243 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:55.619557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.619741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.620146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:55.620146 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.620146 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:55.620146 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:55.620146 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.620146 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:55.620146 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.620146 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:18:55.620631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:18:55.621036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.621036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:18:55.621036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.621036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:18:55.621036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:18:55.621036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.621036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:18:55.621036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.621036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.621294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:55.621637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:55.622065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:55.622439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.622966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:18:55.633068 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:18:55.633326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:18:55.633453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:18:55.633636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:18:55.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.634157 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:18:55.634376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.634809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:18:55.634809 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.634809 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:18:55.634809 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:18:55.634809 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.634809 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:18:55.634809 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.634809 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.635761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.636139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:18:55.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:18:55.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:18:55.637341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:18:55.637856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:18:55.637856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:18:55.637856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:00.348907 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:00.349381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:00.349500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:00.350298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:00.377779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.378021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.378598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:00.378598 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.378598 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:00.378598 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:00.378598 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.378598 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:00.378598 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.378598 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.379403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.379878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:00.380315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.380315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:00.380529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:00.381012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.381389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:00.381389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.381389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:00.595105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:00.595549 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:00.597046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:00.597046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.597046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.597046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.597186 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.597219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.597305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.597414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.597938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:00.597938 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.597938 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:00.597938 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:00.597938 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.597938 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:00.597938 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.597938 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.598962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.599243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:00.599446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.599446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:00.600061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:00.600390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.600724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:00.600724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.600724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:00.605966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:00.606308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:00.606426 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:00.606553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:00.606917 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.607093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.607577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:00.607577 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.607577 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:00.607577 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:00.607577 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.607577 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:00.607577 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.607577 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:00.608450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:00.608788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.608788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:00.608788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.608788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:00.608788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:00.608788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.608788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:00.608788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.608788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.609324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:00.609614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:00.610018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.610018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:00.610018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.610018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:00.610378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.610682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:00.610975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.610975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:00.610975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:00.610975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.610975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:00.610975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.610975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:00.619267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:00.619589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:00.619715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:00.619860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:00.620236 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.620427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.620901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:00.620901 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.620901 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:00.620901 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:00.620901 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.620901 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:00.620901 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.620901 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:00.621749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:00.622012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.622012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:00.622012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.622012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:00.622012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:00.622012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.622012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:00.622012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.622012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.622634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:00.622856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:00.623070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.623070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:00.623070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.623070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.623794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:00.624170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:00.624521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.624521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:00.624521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.624521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:00.633404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:00.633658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:00.633776 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:00.633956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:00.634250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.634427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:00.634598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.635036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:00.635036 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.635036 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:00.635036 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:00.635036 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.635036 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:00.635036 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.635036 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.635996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.636234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:00.636440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.636440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:00.637005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:00.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:00.638051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:00.638051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:00.638051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:05.349392 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:05.349992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:05.349992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:05.350452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:05.378323 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.378601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.379317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:05.379317 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.379317 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:05.379317 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:05.379317 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.379317 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:05.379317 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.379317 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.380070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.380359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:05.380581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.380581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:05.380766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:05.381319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.381646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:05.381646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.381646 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:05.595536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:05.595807 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:05.595925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:05.596648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.596808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.596926 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.597023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.597132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.597218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.597362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.597810 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:05.597810 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.597810 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:05.597810 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:05.597810 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.597810 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:05.597810 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.597810 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.598426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.598815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:05.599154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:05.599580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.599990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:05.599990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:05.599990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.599990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:05.599990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.599990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:05.606268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:05.606477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:05.606588 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:05.606744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:05.607038 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.607178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.607536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:05.607536 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.607536 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:05.607536 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:05.607536 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.607536 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:05.607536 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.607536 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:05.608152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:05.608399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.608399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:05.608399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.608399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:05.608399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:05.608399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.608399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:05.608399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.608399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.608918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:05.609163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:05.609377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.609377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:05.609377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.609377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:05.609847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.610164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:05.610496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.610496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:05.610496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:05.610496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.610496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:05.610496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.610496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:05.619575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:05.619857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:05.619967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:05.620104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:05.620393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.620597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.621117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:05.621117 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.621117 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:05.621117 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:05.621117 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.621117 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:05.621117 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.621117 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:05.621842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:05.622099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.622099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:05.622099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.622099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:05.622099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:05.622099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.622099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:05.622099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.622099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.622752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:05.622972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:05.623171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.623171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:05.623171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.623171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.623875 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:05.624177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:05.624451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.624451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:05.624451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.624451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:05.633708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:05.633954 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:05.634101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:05.634278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:05.634537 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.634766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:05.634958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.636452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:05.636452 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.636452 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:05.636452 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:05.636452 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.636452 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:05.636452 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.636452 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.637137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.637413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:05.637614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.637614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:05.638230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:05.638523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:05.638980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:05.638980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:05.638980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:10.349777 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:10.350213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:10.350213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:10.350742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:10.398396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.398597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.399606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:10.399606 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.399606 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:10.399606 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:10.399606 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.399606 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:10.399606 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.399606 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.402336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.402707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:10.402977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.402977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:10.406785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:10.407165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:10.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:10.595994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:10.596447 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:10.596559 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:10.597402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.597662 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.597821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.597955 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.598088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.598208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.598322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.599454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:10.599454 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.599454 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:10.599454 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:10.599454 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.599454 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:10.599454 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.599454 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.600167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.600532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:10.600904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:10.601172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.601449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:10.601449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:10.601449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.601449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:10.601449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.601449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:10.606549 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:10.606715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:10.606797 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:10.606887 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:10.607188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.607356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.607791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:10.607791 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.607791 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:10.607791 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:10.607791 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.607791 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:10.607791 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.607791 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:10.608303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:10.608598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.608598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:10.608598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.608598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:10.608598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:10.608598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.608598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:10.608598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.608598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.608884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:10.609111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:10.609360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.609360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:10.609360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.609360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:10.609538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.609833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:10.610185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.610185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:10.610185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:10.610185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.610185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:10.610185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.610185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:10.620168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:10.620259 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:10.620385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:10.620734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.620947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.621379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:10.621379 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.621379 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:10.621379 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:10.621379 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.621379 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:10.621379 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.621379 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:10.621929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:10.622178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.622178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:10.622178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.622178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:10.622178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:10.622178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.622178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:10.622178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.622178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.622539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:10.622750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:10.623011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.623011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:10.623011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.623011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.623293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:10.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:10.623975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.623975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:10.623975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.623975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:10.634037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:10.634223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:10.634304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:10.634452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:10.634763 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.634964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:10.635158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.635565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:10.635565 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.635565 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:10.635565 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:10.635565 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.635565 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:10.635565 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.635565 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.636134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.636369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:10.636612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.636612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:10.636897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:10.637190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:10.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:10.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:10.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:15.350293 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:15.351348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:15.351348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:15.351348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:15.396372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.396596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.397209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:15.397209 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.397209 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:15.397209 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:15.397209 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.397209 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:15.397209 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.397209 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.397892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.398152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:15.398406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.398406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:15.398566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:15.398876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.399211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:15.399211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.399211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:15.596600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:15.596797 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:15.596838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:15.597701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.597888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.598002 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.598106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.598207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.598312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.598429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.598905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:15.598905 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.598905 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:15.598905 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:15.598905 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.598905 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:15.598905 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.598905 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.599581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.599848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:15.600056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.600056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:15.600267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:15.600573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.600877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:15.600877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.600877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:15.606761 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:15.606902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:15.606980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:15.607072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:15.607445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.607633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.608044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:15.608044 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.608044 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:15.608044 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:15.608044 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.608044 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:15.608044 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.608044 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:15.608557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:15.608886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.608886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:15.608886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.608886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:15.608886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:15.608886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.608886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:15.608886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.608886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.609170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:15.609415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:15.609664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.609664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:15.609664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.609664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:15.609884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.610236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:15.610535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.610535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:15.610535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:15.610535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.610535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:15.610535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.610535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:15.620189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:15.620329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:15.620410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:15.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:15.620940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.621141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.621585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:15.621585 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.621585 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:15.621585 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:15.621585 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.621585 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:15.621585 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.621585 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:15.622080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:15.622417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.622417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:15.622417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.622417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:15.622417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:15.622417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.622417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:15.622417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.622417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.622653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:15.622872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:15.623093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.623093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:15.623093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.623093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.623347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:15.623681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:15.623995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.623995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:15.623995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.623995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:15.634256 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:15.634398 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:15.634477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:15.634587 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:15.635020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:15.635433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.635887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:15.635887 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.635887 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:15.635887 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:15.635887 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.635887 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:15.635887 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.635887 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.636435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.636700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:15.636924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.636924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:15.637048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:15.637383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:15.637691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:15.637691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:15.637691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:20.350716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:20.352913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:20.352913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:20.352913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:20.389596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.389785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:20.391788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:20.392120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:20.392418 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:20.392770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:20.393104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.393104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:20.393104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.393104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:20.393104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:20.393104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.393104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:20.393104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.393104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:20.597003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:20.597313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:20.597488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:20.598376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.598553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.598719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.598866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.599015 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.599160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.599297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:20.601251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:20.601561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:20.601803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.602033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:20.602340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:20.602340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.602340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:20.602340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.602340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:20.602340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:20.602340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.602340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:20.602340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.602340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:20.607043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:20.607276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:20.607472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:20.607672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:20.608016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.608217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.610427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:20.610790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.611009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.611245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:20.611465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:20.611732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:20.612026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.612026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:20.612026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.612026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:20.612026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:20.612026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.612026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:20.612026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.612026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:20.620614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:20.620964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:20.621189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:20.621373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:20.621786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.621976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.625296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:20.625853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.626102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.626326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:20.626527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:20.626840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:20.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:20.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:20.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:20.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:20.635060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:20.635060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:20.635060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:20.635060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:20.635354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.635593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:20.635825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:20.637630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:20.638094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.638486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:20.638486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:20.638486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.638486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:20.638486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.638486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:20.639024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:20.639399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:20.639801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:20.639801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:20.639801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:25.353759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:25.353759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:25.353759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:25.353759 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:25.385067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.385406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:25.387677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:25.387995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:25.388222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:25.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:25.388783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.388783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:25.388783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.388783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:25.388783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:25.388783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.388783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:25.388783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.388783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:25.597590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:25.598535 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:25.598535 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:25.598957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.599198 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.599347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.599468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.599579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.599699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.599805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:25.601649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:25.601973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:25.602271 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.602517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:25.602868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:25.602868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.602868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:25.602868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.602868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:25.602868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:25.602868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.602868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:25.602868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.602868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:25.607335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:25.607562 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:25.607726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:25.607877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:25.608285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.608437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:25.611057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.611311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.611540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:25.611769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:25.612038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:25.612374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.612374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:25.612374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.612374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:25.612374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:25.612374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.612374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:25.612374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.612374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:25.621056 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:25.621289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:25.621434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:25.621597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:25.622034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.622232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.624548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:25.624901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.625414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:25.625639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:25.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.626256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:25.626256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.626256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:25.626256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:25.626256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.626256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:25.626256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.626256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:25.635086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:25.635708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:25.635708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:25.635708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:25.635950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.636131 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:25.636307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:25.638087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:25.638369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:25.638659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:25.638866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:25.639199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:25.639199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:25.639199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:25.639199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:25.639199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:25.639199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:25.639199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:25.639199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:25.639199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:25.639199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:30.352942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:30.353261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:30.353301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:30.354197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:30.381295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.381520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.382062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:30.382062 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.382062 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:30.382062 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:30.382062 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.382062 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:30.382062 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.382062 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.382745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.383021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:30.383245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.383245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:30.383521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:30.383827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:30.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.384153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:30.598135 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:30.598671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:30.598771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:30.599573 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.599786 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.599900 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.600018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.600129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.600257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.600362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.600885 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:30.600885 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.600885 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:30.600885 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:30.600885 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.600885 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:30.600885 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.600885 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.601817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:30.602061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.602061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:30.602324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:30.602795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.603339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:30.603339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.603339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:30.607826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:30.608117 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:30.608227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:30.608337 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:30.608688 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.608873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.609319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:30.609319 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.609319 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:30.609319 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:30.609319 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.609319 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:30.609319 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.609319 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:30.610112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:30.610419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.610419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:30.610419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.610419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:30.610419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:30.610419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.610419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:30.610419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.610419 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.610998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:30.611258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:30.611481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.611481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:30.611481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.611481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:30.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.612473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:30.612827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.612827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:30.612827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:30.612827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.612827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:30.612827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.612827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:30.621404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:30.621671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:30.621770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:30.621876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:30.622191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.622400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.622836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:30.622836 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.622836 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:30.622836 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:30.622836 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.622836 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:30.622836 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.622836 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:30.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:30.623617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.623617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:30.623617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.623617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:30.623617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:30.623617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.623617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:30.623617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.623617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.623990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:30.624211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:30.624437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.624437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:30.624437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.624437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:30.624983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:30.625269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.625269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:30.625269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.625269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:30.635484 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:30.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:30.635866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:30.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:30.636355 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.636564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:30.636771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.637188 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:30.637188 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.637188 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:30.637188 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:30.637188 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.637188 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:30.637188 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.637188 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.637873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.638233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:30.638453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.638453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:30.638640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:30.638956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:30.639261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:30.639261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:30.639261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:35.353356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:35.353844 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:35.353844 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:35.354438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:35.386889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.387154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.387859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:35.387859 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.387859 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:35.387859 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:35.387859 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.387859 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:35.387859 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.387859 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.388673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.388948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:35.389173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.389173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:35.389434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:35.389772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.390175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:35.390175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.390175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:35.598703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:35.598967 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:35.599088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:35.599866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.600037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.600174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.600308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.600439 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.600563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.600721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.601220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:35.601220 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.601220 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:35.601220 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:35.601220 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.601220 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:35.601220 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.601220 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.601864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.602111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:35.602328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.602328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:35.602527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:35.602837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.603174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:35.603174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.603174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:35.608217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:35.608447 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:35.608533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:35.608688 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:35.609029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.609157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.609566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:35.609566 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.609566 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:35.609566 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:35.609566 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.609566 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:35.609566 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.609566 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:35.610067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:35.610359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.610359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:35.610359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.610359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:35.610359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:35.610359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.610359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:35.610359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.610359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.610616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:35.610878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:35.611108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.611108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:35.611108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.611108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:35.611276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.611560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:35.611911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.611911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:35.611911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:35.611911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.611911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:35.611911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.611911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:35.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:35.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:35.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:35.622612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:35.622960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.623174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.623660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:35.623660 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.623660 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:35.623660 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:35.623660 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.623660 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:35.623660 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.623660 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:35.624346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:35.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:35.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:35.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:35.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:35.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.624606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.625192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:35.625399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:35.625677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.625677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:35.625677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.625677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.626156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:35.626459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:35.626794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.626794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:35.626794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.626794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:35.635874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:35.636191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:35.636307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:35.636470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:35.636851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.637078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:35.637300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.637927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:35.637927 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.637927 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:35.637927 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:35.637927 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.637927 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:35.637927 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.637927 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.638798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.639014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:35.639331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.639331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:35.639844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:35.640100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:35.640403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:35.640403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:35.640403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:40.353801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:40.354341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:40.354341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:40.355063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:40.383944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.383944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.384155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:40.384155 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.384155 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:40.384155 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:40.384155 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.384155 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:40.384155 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.384155 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.384926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.385192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:40.385436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.385436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:40.385789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:40.386063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.386427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:40.386427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.386427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:40.599359 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:40.600378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:40.600515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:40.601648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.602477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.602477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.605520 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.606619 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.606619 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.606619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.606849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:40.606849 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.606849 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:40.606849 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:40.606849 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.606849 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:40.606849 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.606849 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.607855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.608121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:40.608351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.608351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:40.609048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:40.609322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.609676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:40.609676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.609676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:40.609676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:40.609676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:40.609676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:40.610010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:40.610264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.610478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.611343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:40.611343 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.611343 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:40.611343 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:40.611343 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.611343 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:40.611343 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.611343 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:40.612195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:40.612472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.612472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:40.612472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.612472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:40.612472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:40.612472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.612472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:40.612472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.612472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.612987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:40.613190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:40.613384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.613384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:40.613384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.613384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:40.613892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.614207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:40.614527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.614527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:40.614527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:40.614527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.614527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:40.614527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.614527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:40.624652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:40.624652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:40.624652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:40.625109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.625357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.625773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:40.625773 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.625773 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:40.625773 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:40.625773 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.625773 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:40.625773 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.625773 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:40.626635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:40.626889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.626889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:40.626889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.626889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:40.626889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:40.626889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.626889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:40.626889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.626889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.627253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:40.627478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:40.627709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.627709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:40.627709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.627709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.627989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:40.628287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:40.628592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.628592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:40.628592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.628592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:40.636450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:40.636822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:40.636933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:40.637116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:40.637452 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.637708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:40.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.638434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:40.638434 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.638434 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:40.638434 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:40.638434 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.638434 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:40.638434 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.638434 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.639360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.639584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:40.639802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.639802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:40.640521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:40.640791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:40.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:40.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:40.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:45.354183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:45.354552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:45.354712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:45.355386 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:45.388462 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.388660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.389148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:45.389148 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.389148 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:45.389148 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:45.389148 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.389148 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:45.389148 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.389148 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.389804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.390110 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:45.390340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.390340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:45.390645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:45.390950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.391259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:45.391259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.391259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:45.600502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:45.600799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:45.600910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:45.601586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.601742 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.601851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.601953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.602045 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.602138 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.602249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.602659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:45.602659 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.602659 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:45.602659 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:45.602659 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.602659 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:45.602659 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.602659 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.603320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.603534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:45.603758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.603758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:45.604446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:45.604773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.605082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:45.605082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.605082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:45.609653 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:45.609842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:45.609928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:45.610017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:45.610225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.610353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.610791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:45.610791 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.610791 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:45.610791 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:45.610791 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.610791 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:45.610791 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.610791 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:45.611295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:45.611561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.611561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:45.611561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.611561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:45.611561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:45.611561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.611561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:45.611561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.611561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.611961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:45.612166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:45.612376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.612376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:45.612376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.612376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:45.612967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.613245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:45.613511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.613511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:45.613511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:45.613511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.613511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:45.613511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.613511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:45.623740 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:45.623934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:45.624020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:45.624116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:45.624369 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.624545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.624868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:45.624868 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.624868 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:45.624868 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:45.624868 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.624868 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:45.624868 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.624868 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:45.625350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:45.625616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.625616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:45.625616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.625616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:45.625616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:45.625616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.625616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:45.625616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.625616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.625914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:45.626133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:45.626381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.626381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:45.626381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.626381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.626583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:45.626904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:45.627204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.627204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:45.627204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.627204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:45.636822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:45.637146 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:45.637285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:45.637442 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:45.638962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.639142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:45.639325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.639786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:45.639786 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.639786 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:45.639786 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:45.639786 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.639786 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:45.639786 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.639786 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.640517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.640769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:45.640978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.640978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:45.641281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:45.641547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:45.641860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:45.641860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:45.641860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:50.354669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:50.355198 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:50.355198 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:50.355697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:50.380313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.380514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.381021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:50.381021 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.381021 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:50.381021 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:50.381021 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.381021 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:50.381021 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.381021 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.381850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.382352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:50.382800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.382800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:50.383009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:50.383324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.383665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:50.383665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.383665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:50.600857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:50.601140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:50.601244 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:50.602227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.602482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.602671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.602816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.602944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.603090 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.603214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.603801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:50.603801 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.603801 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:50.603801 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:50.603801 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.603801 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:50.603801 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.603801 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.604512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.604869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:50.605083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.605083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:50.605254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:50.605506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.605821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:50.605821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.605821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:50.609879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:50.610094 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:50.610208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:50.610315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:50.610618 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.610780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.611134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:50.611134 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.611134 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:50.611134 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:50.611134 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.611134 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:50.611134 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.611134 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:50.611736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:50.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:50.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:50.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:50.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:50.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.612173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.612566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:50.612922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:50.613303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.613303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:50.613303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.613303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:50.613614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.614068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:50.614528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.614528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:50.614528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:50.614528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.614528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:50.614528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.614528 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:50.623965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:50.624167 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:50.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:50.624441 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:50.624669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.624816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.625122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:50.625122 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.625122 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:50.625122 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:50.625122 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.625122 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:50.625122 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.625122 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:50.625724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:50.626122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.626122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:50.626122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.626122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:50.626122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:50.626122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.626122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:50.626122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.626122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.626554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:50.626869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:50.627227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.627227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:50.627227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.627227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.627581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:50.628146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:50.628664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.628664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:50.628664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.628664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:50.637100 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:50.637310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:50.637418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:50.637578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:50.637837 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.638005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:50.638178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.638557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:50.638557 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.638557 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:50.638557 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:50.638557 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.638557 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:50.638557 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.638557 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.639213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.639429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:50.639653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.639653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:50.640071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:50.640330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:50.640679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:50.640679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:50.640679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:55.355019 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:55.355418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:55.355519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:55.356246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:55.363585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=243 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:19:55.382787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.382986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.383663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:55.383663 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.383663 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:55.383663 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:55.383663 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.383663 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:55.383663 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.383663 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.384325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.384585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:55.384819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.384819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:55.385253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:55.385674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.386064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:55.386064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.386064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:55.601303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:55.601456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:55.601579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:55.602472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.602720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.602900 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.603055 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.603173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.603285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.603407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.603916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:55.603916 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.603916 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:55.603916 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:55.603916 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.603916 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:55.603916 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.603916 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.604597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.605010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:55.605222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.605222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:55.605326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:55.605651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.605994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:55.605994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.605994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:55.610107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:55.610260 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:55.610348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:55.610436 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:19:55.610884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.611081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.611596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:55.611596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.611596 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:55.611596 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:55.611596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.611596 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:55.611596 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.611596 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:55.612144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:55.612421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.612421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:55.612421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.612421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:55.612421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:55.612421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.612421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:55.612421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.612421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.612690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:55.612931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:55.613132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.613132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:55.613132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.613132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:55.613330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.613604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:55.613947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.613947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:55.613947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:55.613947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.613947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:55.613947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.613947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:55.624318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:55.624550 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:55.624656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:55.624765 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:55.625190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.625413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:55.625925 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.625925 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:55.625925 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:55.625925 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.625925 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:55.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.625925 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:19:55.626473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:19:55.626740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.626740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:19:55.626740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.626740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:19:55.626740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:19:55.626740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.626740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:19:55.626740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.626740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.627075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:55.627281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:55.627491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.627491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:55.627491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.627491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.627844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:55.628150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:55.628439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.628439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:55.628439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.628439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:19:55.637483 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:19:55.637729 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:19:55.637817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:19:55.637959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:19:55.638330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.638550 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:19:55.638777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.639240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:19:55.639240 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.639240 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:19:55.639240 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:19:55.639240 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.639240 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:19:55.639240 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.639240 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.639900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.640124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:19:55.640309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.640309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:19:55.640563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:19:55.640832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:19:55.641179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:19:55.641179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:19:55.641179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:00.355438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:00.355899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:00.355899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:00.356757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:00.393202 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.393367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.393925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:00.393925 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.393925 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:00.393925 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:00.393925 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.393925 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:00.393925 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.393925 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.394550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.394960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:00.395275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.395275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:00.395742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:00.396305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.396957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:00.396957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.396957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:00.601695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:00.601943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:00.602041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:00.602889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.603069 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.603186 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.603304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.603417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.603523 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.603633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.604098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:00.604098 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.604098 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:00.604098 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:00.604098 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.604098 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:00.604098 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.604098 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.604832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.605063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:00.605270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.605270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:00.605537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:00.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.606153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:00.606153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.606153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:00.610389 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:00.610594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:00.610708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:00.610799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:00.611098 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.611238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.611633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:00.611633 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.611633 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:00.611633 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:00.611633 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.611633 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:00.611633 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.611633 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:00.612211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:00.612456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.612456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:00.612456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.612456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:00.612456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:00.612456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.612456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:00.612456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.612456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.612842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:00.613119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:00.613337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.613337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:00.613337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.613337 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:00.613572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.613871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:00.614167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.614167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:00.614167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:00.614167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.614167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:00.614167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.614167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:00.624851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:00.625155 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:00.625253 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:00.625357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:00.625743 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.625898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.626316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:00.626316 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.626316 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:00.626316 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:00.626316 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.626316 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:00.626316 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.626316 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:00.627255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:00.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:00.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:00.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:00.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:00.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.627495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.628047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:00.628409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:00.628730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.628730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:00.628730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.628730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.629015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:00.629978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:00.630615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.630615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:00.630615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.630615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:00.638007 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:00.638300 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:00.638404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:00.638563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:00.638851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.638991 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:00.639153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.639580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:00.639580 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.639580 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:00.639580 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:00.639580 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.639580 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:00.639580 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.639580 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.640606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.640854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:00.641041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.641041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:00.641485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:00.641766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:00.642048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:00.642048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:00.642048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:05.355857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:05.357391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:05.357391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:05.357391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:05.392480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.392721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.393248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:05.393248 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.393248 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:05.393248 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:05.393248 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.393248 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:05.393248 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.393248 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.394168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.394716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:05.395147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.395147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:05.395223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:05.395545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.395863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:05.395863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.395863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:05.602170 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:05.602632 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:05.602736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:05.603495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.603729 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.603867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.603991 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.604127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.604262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.604371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.604867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:05.604867 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.604867 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:05.604867 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:05.604867 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.604867 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:05.604867 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.604867 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.605697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.605903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:05.606091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.606091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:05.606374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:05.606661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.606933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:05.606933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.606933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:05.610830 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:05.611080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:05.611165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:05.611266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:05.611606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.611820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.612285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:05.612285 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.612285 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:05.612285 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:05.612285 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.612285 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:05.612285 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.612285 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:05.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:05.613078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.613078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:05.613078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.613078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:05.613078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:05.613078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.613078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:05.613078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.613078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.613369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:05.613772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:05.614141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.614373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.614680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:05.625347 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:05.625689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:05.625788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:05.625888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:05.626553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.626553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.626982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:05.626982 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.626982 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:05.626982 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:05.626982 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.626982 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:05.626982 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.626982 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:05.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:05.627820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.627820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:05.627820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.627820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:05.627820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:05.627820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.627820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:05.627820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.627820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.628171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:05.628365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:05.628576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.628576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:05.628576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.628576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.628917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:05.629193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:05.629471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.629471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:05.629471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.629471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:05.638388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:05.638658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:05.638762 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:05.638889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:05.639302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.639594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:05.639867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.640559 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:05.640559 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.640559 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:05.640559 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:05.640559 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.640559 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:05.640559 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.640559 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.641030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:05.641435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.641435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:05.641702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:05.641962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:05.642228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:05.642228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:05.642228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:10.356255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:10.356713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:10.356713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:10.357290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:10.387639 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.387820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.388319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:10.388319 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.388319 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:10.388319 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:10.388319 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.388319 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:10.388319 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.388319 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.389136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.389407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:10.389658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.389658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:10.389966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:10.390252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.390595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:10.390595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.390595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:10.602643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:10.602957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:10.603062 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:10.603892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.604074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.604190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.604308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.604430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.604539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.604663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.605124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:10.605124 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.605124 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:10.605124 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:10.605124 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.605124 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:10.605124 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.605124 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.605855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.606137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:10.606396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.606396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:10.606577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:10.606894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.607228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:10.607228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.607228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:10.611127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:10.611314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:10.611415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:10.611510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:10.611840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.612026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.612463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:10.612463 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.612463 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:10.612463 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:10.612463 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.612463 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:10.612463 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.612463 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:10.613020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:10.613319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.613319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:10.613319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.613319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:10.613319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:10.613319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.613319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:10.613319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.613319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.613574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:10.613839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:10.614070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.614070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:10.614070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.614070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:10.614279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.614659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:10.614978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.614978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:10.614978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:10.614978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.614978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:10.614978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.614978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:10.625710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:10.625941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:10.626051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:10.626160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:10.626524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.626760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:10.627214 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.627214 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:10.627214 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:10.627214 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.627214 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:10.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.627214 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:10.627823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:10.628259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.628259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:10.628259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.628259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:10.628259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:10.628259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.628259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:10.628259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.628259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.628513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:10.628787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:10.629006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.629006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:10.629006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.629006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.629307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:10.629641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:10.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:10.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:10.638697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:10.638876 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:10.638988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:10.639123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:10.639495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.639693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:10.640282 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.640282 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:10.640282 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:10.640282 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.640282 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:10.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.640917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.641167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:10.641409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.641409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:10.641681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:10.641966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:10.642281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:10.642281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:10.642281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:15.356690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:15.358402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:15.358402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:15.358402 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:15.392840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.393052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.393586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:15.393586 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.393586 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:15.393586 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:15.393586 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.393586 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:15.393586 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.393586 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.394371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.394636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:15.394839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.394839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:15.395083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:15.395360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.395696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:15.395696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.395696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:15.603124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:15.603434 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:15.603542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:15.604524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.604765 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.604921 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.605071 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.605216 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.605358 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.605497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.606160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:15.606160 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.606160 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:15.606160 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:15.606160 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.606160 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:15.606160 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.606160 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.607030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.607483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:15.607815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.607815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:15.608203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:15.608638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.609276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:15.609276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.609276 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:15.611381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:15.611588 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:15.611690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:15.612010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:15.612254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.612466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.613079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:15.613079 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.613079 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:15.613079 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:15.613079 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.613079 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:15.613079 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.613079 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:15.613758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:15.614111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.614111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:15.614111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.614111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:15.614111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:15.614111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.614111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:15.614111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.614111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.614398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:15.615029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:15.615303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.615558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:15.625990 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:15.626208 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:15.626312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:15.626445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:15.626948 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.627223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.627746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:15.627746 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.627746 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:15.627746 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:15.627746 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.627746 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:15.627746 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.627746 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:15.628453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:15.628772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.628772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:15.628772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.628772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:15.628772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:15.628772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.628772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:15.628772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.628772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.629144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:15.629364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:15.629689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.629689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:15.629689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.629689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.630178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:15.630531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:15.630899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.630899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:15.630899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.630899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:15.639035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:15.639279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:15.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:15.639546 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:15.639858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.640080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:15.640322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.640814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:15.640814 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.640814 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:15.640814 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:15.640814 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.640814 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:15.640814 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.640814 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.641468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.641722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:15.641982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.641982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:15.642227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:15.642533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:15.642918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:15.642918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:15.642918 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:20.357655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:20.358137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:20.358137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:20.358640 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:20.392426 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.392691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.393391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:20.393391 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.393391 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:20.393391 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:20.393391 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.393391 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:20.393391 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.393391 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.394090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.394434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:20.394764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.394764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:20.394824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:20.395291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.395743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:20.395743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.395743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:20.603552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:20.603820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:20.603939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:20.604922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.605206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.605370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.605503 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.605633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.605775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.605907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.606517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:20.606517 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.606517 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:20.606517 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:20.606517 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.606517 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:20.606517 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.606517 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.607411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.607698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:20.607913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.607913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:20.608335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:20.608697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.608997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:20.608997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.608997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:20.611640 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:20.611865 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:20.611973 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:20.612089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:20.612511 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.612749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.613311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:20.613311 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.613311 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:20.613311 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:20.613311 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.613311 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:20.613311 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.613311 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:20.613999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:20.614286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.614286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:20.614286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.614286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:20.614286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:20.614286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.614286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:20.614286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.614286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.614720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:20.615080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:20.615474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.615474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:20.615474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.615474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:20.615597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.616070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:20.616543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.616543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:20.616543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:20.616543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.616543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:20.616543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.616543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:20.626451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:20.626836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:20.626985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:20.627166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:20.627736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.628069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.629445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:20.629445 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.629445 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:20.629445 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:20.629445 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.629445 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:20.629445 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.629445 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:20.631800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:20.632262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.632262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:20.632262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.632262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:20.632262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:20.632262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.632262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:20.632262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.632262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.632556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:20.632861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:20.633396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.633396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:20.633396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.633396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.633706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:20.634238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:20.634814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.634814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:20.634814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.634814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:20.639428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:20.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:20.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:20.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:20.640585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.640877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:20.641141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.641802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:20.641802 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.641802 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:20.641802 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:20.641802 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.641802 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:20.641802 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.641802 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.642751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.643064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:20.643390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.643390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:20.643652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:20.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:20.644427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:20.644427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:20.644427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:25.357988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:25.358502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:25.358570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:25.359384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:25.389121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.389302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.389770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:25.389770 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.389770 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:25.389770 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:25.389770 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.389770 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:25.389770 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.389770 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.390507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.390961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:25.391377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.391377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:25.391597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:25.391969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.392278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:25.392278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.392278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:25.605613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:25.605613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:25.605613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:25.606669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.606669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.606669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.606669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.606888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.606888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.606973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.607730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:25.607730 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.607730 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:25.607730 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:25.607730 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.607730 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:25.607730 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.607730 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.608156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.608435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:25.608675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.608675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:25.609167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:25.609508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.609903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:25.609903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.609903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:25.612838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:25.612838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:25.612838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:25.612838 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:25.613076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.613183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.613723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:25.613723 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.613723 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:25.613723 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:25.613723 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.613723 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:25.613723 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.613723 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:25.614258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:25.614547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.614547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:25.614547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.614547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:25.614547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:25.614547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.614547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:25.614547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.614547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.615205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:25.615508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:25.615794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.616042 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.616356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:25.629099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:25.629427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:25.629526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:25.629652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:25.629972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.630181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.630637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:25.630637 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.630637 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:25.630637 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:25.630637 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.630637 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:25.630637 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.630637 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:25.631131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:25.631401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.631401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:25.631401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.631401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:25.631401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:25.631401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.631401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:25.631401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.631401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.631740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:25.631968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:25.632216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.632216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:25.632216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.632216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.632490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:25.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:25.633175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.633175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:25.633175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.633175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:25.640069 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:25.640404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:25.640490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:25.640631 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:25.640944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.641161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:25.641388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.641845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:25.641845 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.641845 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:25.641845 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:25.641845 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.641845 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:25.641845 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.641845 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.642535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.642815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:25.643143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.643143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:25.643413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:25.643700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:25.644031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:25.644031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:25.644031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:30.358409 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:30.358869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:30.358946 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:30.359500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:30.389420 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.389635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.390207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:30.390207 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.390207 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:30.390207 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:30.390207 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.390207 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:30.390207 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.390207 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.390851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.391063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:30.391290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.391290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:30.391589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:30.391872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.392156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:30.392156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.392156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:30.605424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:30.605710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:30.605814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:30.606643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.606801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.606935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.607026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.607151 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.607283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.607376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.607864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:30.607864 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.607864 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:30.607864 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:30.607864 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.607864 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:30.607864 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.607864 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.608795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.609155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:30.609442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.609442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:30.609839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:30.610118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.610430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:30.610430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.610430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:30.612380 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:30.612569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:30.612695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:30.612817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:30.613115 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.613293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.613717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:30.613717 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.613717 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:30.613717 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:30.613717 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.613717 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:30.613717 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.613717 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:30.614541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:30.614840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.614840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:30.614840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.614840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:30.614840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:30.614840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.614840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:30.614840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.614840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.615280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:30.615499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:30.615888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.615888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:30.615888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.615888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:30.616305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.616616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:30.616910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.616910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:30.616910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:30.616910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.616910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:30.616910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.616910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:30.629680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:30.629936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:30.630098 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:30.630257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:30.631514 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.631514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.631773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:30.631773 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.631773 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:30.631773 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:30.631773 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.631773 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:30.631773 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.631773 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:30.632535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:30.632997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.632997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:30.632997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.632997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:30.632997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:30.632997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.632997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:30.632997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.632997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.633405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:30.633720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:30.634068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.634068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:30.634068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.634068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:30.634706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:30.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:30.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:30.640471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:30.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:30.642137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:30.642321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:30.642795 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.642988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:30.643164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.643757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:30.643757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.643757 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:30.643757 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:30.643757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.643757 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:30.643757 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.643757 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.644577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.644939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:30.645306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.645306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:30.645502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:30.646677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:30.647191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:30.647191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:30.647191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:35.359120 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:35.359699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:35.359699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:35.360270 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:35.402976 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.403239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.403820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:35.403820 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.403820 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:35.403820 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:35.403820 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.403820 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:35.403820 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.403820 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.404732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.405078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:35.405400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.405400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:35.405485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:35.405921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.406404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:35.406404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.406404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:35.605806 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:35.606113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:35.606240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:35.608066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.608066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.608066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.608066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.608066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.608326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.608326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.608326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:35.608326 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.608326 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:35.608326 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:35.608326 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.608326 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:35.608326 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.608326 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.609101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.609331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:35.609535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.609535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:35.610151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:35.610705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.611156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:35.611156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.611156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:35.612616 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:35.613177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:35.613177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:35.613177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:35.613372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.613518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.616164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:35.616636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.616974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.617244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:35.617535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:35.617860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:35.618251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.618251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:35.618251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.618251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:35.618251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:35.618251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.618251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:35.618251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.618251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:35.630013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:35.630358 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:35.630572 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:35.630811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:35.631216 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.631415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.634232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:35.634672 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.635033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:35.635610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:35.635973 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.636378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:35.636378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.636378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:35.636378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:35.636378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.636378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:35.636378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.636378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:35.641158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:35.641430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:35.641594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:35.641827 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:35.642219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.642455 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:35.642690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:35.644437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:35.644820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:35.645106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:35.645393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:35.645908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:35.645908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:35.645908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:35.645908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:35.645908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:35.645908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:35.645908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:35.645908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:35.645908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:35.645908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:40.359570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:40.360114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:40.360114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:40.360863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:40.391019 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.391291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.391700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:40.391700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.391700 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:40.391700 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:40.391700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.391700 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:40.391700 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.391700 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.392403 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.392725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:40.392997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.392997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:40.393377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:40.394037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.394427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:40.394427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.394427 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:40.608874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:40.609254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:40.609432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:40.610265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.610492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.610658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.610808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.610952 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.611079 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.611229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.611834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:40.611834 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.611834 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:40.611834 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:40.611834 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.611834 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:40.611834 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.611834 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.612864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:40.613100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.613100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:40.613548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:40.613876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.614596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:40.614596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.614596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:40.614596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:40.614596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:40.614596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:40.614596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:40.614596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.614757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.615137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:40.615137 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.615137 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:40.615137 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:40.615137 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.615137 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:40.615137 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.615137 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:40.615669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:40.615920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.615920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:40.615920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.615920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:40.615920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:40.615920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.615920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:40.615920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.615920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.616318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:40.616538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:40.616754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.616754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:40.616754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.616754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:40.617056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.617511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:40.617838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.617838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:40.617838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:40.617838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.617838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:40.617838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.617838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:40.630472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:40.630825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:40.630957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:40.631093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:40.631569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.631788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.632333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:40.632333 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.632333 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:40.632333 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:40.632333 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.632333 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:40.632333 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.632333 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:40.633245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:40.633544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.633544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:40.633544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.633544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:40.633544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:40.633544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.633544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:40.633544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.633544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.634100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:40.634442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:40.634741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.634741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:40.634741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.634741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.635197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:40.635477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:40.635792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.635792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:40.635792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.635792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:40.641574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:40.641870 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:40.641975 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:40.642194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:40.642638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.642848 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:40.643056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.643546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:40.643546 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.643546 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:40.643546 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:40.643546 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.643546 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:40.643546 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.643546 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.644540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.644818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:40.645026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.645026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:40.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:40.645897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:40.646183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:40.646183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:40.646183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:45.360094 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:45.360482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:45.360597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:45.361266 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:45.386708 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.386902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.387351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:45.387351 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.387351 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:45.387351 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:45.387351 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.387351 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:45.387351 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.387351 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.388053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.388436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:45.388804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.388804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:45.389106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:45.389551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.390100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:45.390100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.390100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:45.609307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:45.609696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:45.609820 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:45.610696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.610879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.610997 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.611121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.611232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.611333 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.611445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.611925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:45.611925 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.611925 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:45.611925 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:45.611925 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.611925 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:45.611925 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.611925 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.612738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.613147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:45.613546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.613546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:45.614028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:45.614530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.614904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:45.614904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.614904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:45.614904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:45.614904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:45.614904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:45.614904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:45.614904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.615102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.615344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:45.615344 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.615344 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:45.615344 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:45.615344 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.615344 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:45.615344 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.615344 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:45.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:45.616472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.616472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:45.616472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.616472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:45.616472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:45.616472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.616472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:45.616472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.616472 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.616899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:45.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:45.617802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.617802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:45.617802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.617802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:45.617972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.618310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:45.618597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.618597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:45.618597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:45.618597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.618597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:45.618597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.618597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:45.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:45.631191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:45.631318 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:45.631465 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:45.631976 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.632182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.632608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:45.632608 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.632608 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:45.632608 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:45.632608 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.632608 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:45.632608 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.632608 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:45.633343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:45.633677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.633677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:45.633677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.633677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:45.633677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:45.633677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.633677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:45.633677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.633677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.633957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:45.634175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:45.634407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.634407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:45.634407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.634407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.634656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:45.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:45.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:45.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.635312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:45.641920 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:45.642149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:45.642230 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:45.642348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:45.642616 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.642816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:45.642982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.643366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:45.643366 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.643366 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:45.643366 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:45.643366 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.643366 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:45.643366 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.643366 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.643923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.644142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:45.644372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.644372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:45.644563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:45.644833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:45.645158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:45.645158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:45.645158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:50.360557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:50.361008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:50.361044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:50.361757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:50.391855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.392038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.392580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:50.392580 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.392580 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:50.392580 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:50.392580 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.392580 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:50.392580 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.392580 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.393327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.393555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:50.393781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.393781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:50.394079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:50.394347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.394656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:50.394656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.394656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:50.609799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:50.610223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:50.610364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:50.611232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.611417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.611541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.611707 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-65dff2f7-cd03-4707-bea6-8dde9f507d26, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.611811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.611960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.612092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:50.612635 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.612635 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:50.612635 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:50.612635 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.612635 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:50.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.612635 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.613381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.613711 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:50.614035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.614035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:50.614175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:50.614557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.615063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:50.615063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.615063 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:50.615063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:50.615265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:50.615532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:50.615695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:50.616045 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.616215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.616698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:50.616698 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.616698 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:50.616698 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:50.616698 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.616698 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:50.616698 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.616698 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:50.617471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:50.617938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.617938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:50.617938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.617938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:50.617938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:50.617938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.617938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:50.617938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.617938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.618548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:50.618889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:50.619251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.619251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:50.619251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.619251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:50.619995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.620463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:50.620966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.620966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:50.620966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:50.620966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.620966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:50.620966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.620966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:50.631308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:50.631720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:50.631922 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:50.632073 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:50.632512 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.632836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.633831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:50.633831 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.633831 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:50.633831 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:50.633831 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.633831 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:50.633831 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.633831 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:50.634353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:50.634761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.634761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:50.634761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.634761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:50.634761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:50.634761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.634761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:50.634761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.634761 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.635305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:50.635550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:50.635806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.635806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:50.635806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.635806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.636411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:50.636862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:50.637194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.637194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:50.637194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.637194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:50.642637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:50.643033 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:50.643193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:50.643414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:50.643764 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.644027 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:50.644275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.644742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:50.644742 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.644742 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:50.644742 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:50.644742 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.644742 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:50.644742 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.644742 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.645714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.646007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:50.646193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.646193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:50.646851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:50.647155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:50.647425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:50.647425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:50.647425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:55.361145 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:55.362025 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:55.362173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:55.363046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:55.368121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=237 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:20:55.394248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:55.394490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.395116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:55.395116 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.395116 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:55.395116 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:55.395116 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.395116 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:55.395116 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.395116 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.395881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.396181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:55.396461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.396461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:55.396650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:55.396997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.397377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:55.397377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.397377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:55.610290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:55.610553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:55.610707 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:55.611539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:55.611839 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:55.612062 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:55.612116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:55.612251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:55.612356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.612873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:55.612873 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.612873 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:55.612873 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:55.612873 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.612873 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:55.612873 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.612873 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.613734 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.614081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:55.614481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.614481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:55.614638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:55.615048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.615505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:55.615505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.615505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:55.615505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:55.615505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:55.615505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:55.615505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:20:55.615889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:55.616038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.616746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:55.616746 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.616746 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:55.616746 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:55.616746 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.616746 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:55.616746 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.616746 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:55.617667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:55.618073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.618073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:55.618073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.618073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:55.618073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:55.618073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.618073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:55.618073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.618073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.618505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:55.618873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:55.619216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.619216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:55.619216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.619216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:55.619478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.619930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:55.620382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.620382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:55.620382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:55.620382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.620382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:55.620382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.620382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:55.631781 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:55.632214 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:55.632372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:55.632511 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:55.632945 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:55.633142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.633590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:55.633590 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.633590 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:55.633590 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:55.633590 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.633590 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:55.633590 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.633590 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:20:55.634387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:20:55.634790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.634790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:20:55.634790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.634790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:20:55.634790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:20:55.634790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.634790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:20:55.634790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.634790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.635226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:55.635530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:55.635899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.635899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:55.635899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.635899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.636187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:55.636607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:55.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:55.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:20:55.643070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:20:55.643335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:20:55.643427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:20:55.643563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:20:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:55.644127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:20:55.644292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.644895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:20:55.644895 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.644895 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:20:55.644895 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:20:55.644895 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.644895 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:20:55.644895 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.644895 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.645686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.646021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:20:55.646347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.646347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:20:55.646716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:20:55.647108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:20:55.647579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:20:55.647579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:20:55.647579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:00.362211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:00.362766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:00.362766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:00.363210 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:00.391371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:00.391638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.392230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:00.392230 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.392230 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:00.392230 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:00.392230 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.392230 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:00.392230 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.392230 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.392894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.393319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:00.393683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:00.394085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.394613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:00.394613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:00.394613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.394613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:00.394613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.394613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:00.610754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:00.611004 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:00.611097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:00.611794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:00.611929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:00.612121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:00.612152 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:00.612300 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:00.612342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.612799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:00.612799 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.612799 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:00.612799 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:00.612799 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.612799 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:00.612799 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.612799 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.613437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.613679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:00.613927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.613927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:00.614274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:00.614578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.614912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:00.614912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.614912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:00.615357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:00.615553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:00.615661 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:00.615784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:00.616231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:00.616431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.616910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:00.616910 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.616910 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:00.616910 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:00.616910 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.616910 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:00.616910 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.616910 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:00.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:00.617828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.617828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:00.617828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.617828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:00.617828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:00.617828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.617828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:00.617828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.617828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.618183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:00.618479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:00.618824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.619144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:00.632189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:00.632385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:00.632494 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:00.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:00.633056 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:00.633280 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:00.633754 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.633754 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:00.633754 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:00.633754 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.633754 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:00.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:00.634341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:00.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:00.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:00.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:00.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:00.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.635004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:00.635279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:00.635560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.635560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:00.635560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.635560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.635795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:00.636152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:00.636577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.636577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:00.636577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.636577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:00.643553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:00.643846 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:00.643961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:00.644141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:00.644634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:00.644926 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:00.645113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.645765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:00.645765 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.645765 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:00.645765 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:00.645765 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.645765 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:00.645765 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.645765 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.646803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.647100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:00.647320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.647320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:00.647946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:00.648290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:00.648595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:00.648595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:00.648595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:05.362507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:05.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:05.363024 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:05.363369 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:05.390897 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:05.391056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.391483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:05.391483 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.391483 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:05.391483 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:05.391483 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.391483 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:05.391483 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.391483 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.392083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.392344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:05.392659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.392659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:05.392811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:05.393106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.393437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:05.393437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.393437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:05.611159 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:05.611391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:05.611478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:05.612122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:05.612255 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:05.612384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:05.612492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:05.612583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:05.612703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.613116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:05.613116 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.613116 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:05.613116 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:05.613116 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.613116 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:05.613116 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.613116 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.613742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.613987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:05.614193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.614193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:05.614425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:05.614791 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.615107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:05.615107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.615107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:05.615578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:05.615724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:05.615801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:05.615886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:05.616082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:05.616205 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:05.616529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.616529 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:05.616529 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:05.616529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.616529 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:05.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:05.617019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:05.617308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.617308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:05.617308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.617308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:05.617308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:05.617308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.617308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:05.617308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.617308 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.617613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:05.617923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:05.618132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.618132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:05.618132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.618132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:05.618366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.618707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:05.618998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.618998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:05.618998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:05.618998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.618998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:05.618998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.618998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:05.632427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:05.632574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:05.632659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:05.632789 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:05.632988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:05.633135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.633438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:05.633438 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.633438 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:05.633438 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:05.633438 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.633438 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:05.633438 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.633438 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:05.633941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:05.634223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.634223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:05.634223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.634223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:05.634223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:05.634223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.634223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:05.634223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.634223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.634501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:05.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:05.634959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.634959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:05.634959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.634959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.635166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:05.635513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:05.635871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.635871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:05.635871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.635871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:05.643882 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:05.644005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:05.644084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:05.644189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:05.644393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:05.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:05.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:05.645123 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.645123 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:05.645123 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:05.645123 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.645123 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:05.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.645544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.645882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:05.646156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.646156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:05.646212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:05.646970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:05.646970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:05.646970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:10.362994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:10.363417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:10.363417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:10.363881 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:10.392287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:10.392505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.392967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:10.392967 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.392967 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:10.392967 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:10.392967 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.392967 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:10.392967 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.392967 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.393790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.394202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:10.394601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.394601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:10.394917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:10.395233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.395564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:10.395564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.395564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:10.611676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:10.612302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:10.612302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:10.612916 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:10.613710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:10.613710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:10.613710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:10.613710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:10.613929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.614295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:10.614295 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.614295 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:10.614295 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:10.614295 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.614295 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:10.614295 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.614295 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.615074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.615436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:10.615842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.615842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:10.615912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:10.616402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.616953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:10.616953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.616953 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:10.616953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:10.616953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:10.616953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:10.616953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:10.616953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:10.617234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.617376 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:10.617376 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.617376 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:10.617376 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:10.617376 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.617376 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:10.617376 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.617376 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:10.618005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:10.618467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.618467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:10.618467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.618467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:10.618467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:10.618467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.618467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:10.618467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.618467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.618767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:10.619164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:10.619560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.619966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.620498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:10.632798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:10.633116 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:10.633254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:10.633396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:10.633752 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:10.633977 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.634566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:10.634566 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.634566 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:10.634566 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:10.634566 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.634566 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:10.634566 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.634566 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:10.635974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:10.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:10.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:10.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:10.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:10.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.636461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.636942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:10.637312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:10.637739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.637739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:10.637739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.637739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.638354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:10.638755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:10.639112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.639112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:10.639112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.639112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:10.644387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:10.644774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:10.644877 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:10.645024 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:10.645938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:10.646182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:10.646401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.646946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:10.646946 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.646946 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:10.646946 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:10.646946 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.646946 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:10.646946 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.646946 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.647709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.648023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:10.648240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.648240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:10.648393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:10.648910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:10.649452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:10.649452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:10.649452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:15.363265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:15.363720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:15.363720 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:15.364184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:15.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:15.393002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.393641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:15.393641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.393641 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:15.393641 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:15.393641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.393641 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:15.393641 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.393641 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.394334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.394554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:15.394771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.394771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:15.395038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:15.395583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.396153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:15.396153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.396153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:15.612050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:15.612415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:15.612515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:15.613440 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:15.613819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:15.613994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:15.614154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:15.614308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:15.614459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.615077 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:15.615077 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.615077 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:15.615077 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:15.615077 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.615077 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:15.615077 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.615077 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.615820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.616044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:15.616239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.616239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:15.616522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:15.616795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.617097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:15.617097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.617097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:15.617097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:15.617097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:15.617097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:15.617097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:15.617480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:15.617713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:15.618189 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.618189 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:15.618189 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:15.618189 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.618189 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:15.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.618189 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:15.618691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:15.619212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.619739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.619940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:15.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.620367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:15.620653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.620653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:15.620653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:15.620653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.620653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:15.620653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.620653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:15.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:15.633424 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:15.633506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:15.633608 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:15.633980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:15.634217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.634717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:15.634717 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.634717 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:15.634717 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:15.634717 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.634717 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:15.634717 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.634717 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:15.635448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.635448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:15.635448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.635448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:15.635448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:15.635448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.635448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:15.635448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.635448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.635784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:15.636004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:15.636212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.636212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:15.636212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.636212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.636523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:15.637093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:15.637733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.637733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:15.637733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.637733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:15.644736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:15.644985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:15.645092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:15.645226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:15.645528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:15.645767 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:15.645989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.646456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:15.646456 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.646456 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:15.646456 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:15.646456 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.646456 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:15.646456 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.646456 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.647016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.647207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:15.647435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.647435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:15.647647 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:15.648157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:15.648661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:15.648661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:15.648661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:20.363706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:20.364237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:20.364237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:20.365101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:20.397930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:20.398067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.398466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:20.398466 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.398466 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:20.398466 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:20.398466 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.398466 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:20.398466 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.398466 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.399387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.399757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:20.400057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.400057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:20.400439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:20.400866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.401284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:20.401284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.401284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:20.612511 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:20.612939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:20.613090 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:20.613938 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:20.614049 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:20.614169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:20.614276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:20.614376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:20.614491 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.614981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:20.614981 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.614981 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:20.614981 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:20.614981 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.614981 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:20.614981 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.614981 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.615938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.616201 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:20.616441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.616441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:20.617026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:20.617370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.617724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:20.617724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.617724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:20.617724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:20.617724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:20.617724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:20.617849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:20.618107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:20.618270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.618657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:20.618657 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.618657 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:20.618657 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:20.618657 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.618657 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:20.618657 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.618657 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:20.619396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:20.619862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.619862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:20.619862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.619862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:20.619862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:20.619862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.619862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:20.619862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.619862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.620242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:20.620679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:20.621078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.621078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:20.621078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.621078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:20.621239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.621571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:20.621996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.621996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:20.621996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:20.621996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.621996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:20.621996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.621996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:20.633605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:20.633969 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:20.634105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:20.634252 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:20.634570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:20.634818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.635225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:20.635225 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.635225 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:20.635225 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:20.635225 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.635225 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:20.635225 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.635225 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:20.636019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:20.636342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.636342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:20.636342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.636342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:20.636342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:20.636342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.636342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:20.636342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.636342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.636967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:20.637196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:20.637410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.637410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:20.637410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.637410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.638067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:20.638417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:20.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:20.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:20.645133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:20.645394 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:20.645509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:20.645691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:20.645953 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:20.646121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:20.646269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.646666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:20.646666 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.646666 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:20.646666 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:20.646666 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.646666 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:20.646666 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.646666 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.647479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.647762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:20.647956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.647956 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:20.648471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:20.648779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:20.649112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:20.649112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:20.649112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:25.364168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:25.364560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:25.364560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:25.365163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:25.395821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:25.396049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.396668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:25.396668 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.396668 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:25.396668 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:25.396668 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.396668 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:25.396668 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.396668 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.397447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.397751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:25.397995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.397995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:25.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:25.398887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.399176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:25.399176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.399176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:25.612943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:25.613200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:25.613312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:25.614245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:25.614401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:25.614524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-5c02d21a-2948-424b-8546-3197853585c1, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:25.614673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:25.614832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): DelStaticRoutesCommand(_result=None, lrouter=neutron-bbb18b26-58ab-4a30-bfab-291eb339d698, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:25.614974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.615529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:25.615529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.615529 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:25.615529 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:25.615529 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.615529 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:25.615529 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.615529 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.616529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.616807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:25.617046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.617046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:25.617854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:25.618220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.618603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:25.618603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.618603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:25.618603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:25.618823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:25.618977 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:25.619139 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:25.619602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:25.619788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.620247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:25.620247 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.620247 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:25.620247 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:25.620247 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.620247 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:25.620247 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.620247 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:25.621046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:25.621295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.621295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:25.621295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.621295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:25.621295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:25.621295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.621295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:25.621295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.621295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.621881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:25.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:25.622507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.622507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:25.622507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.622507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:25.622923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.623378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:25.623897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.623897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:25.623897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:25.623897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.623897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:25.623897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.623897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:25.633962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:25.634256 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:25.634369 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:25.634510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:25.634985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:25.635219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.635790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:25.635790 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.635790 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:25.635790 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:25.635790 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.635790 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:25.635790 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.635790 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:25.636605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:25.636960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.636960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:25.636960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.636960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:25.636960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:25.636960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.636960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:25.636960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.636960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.637538 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:25.637908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:25.638334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.638334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:25.638334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.638334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.638635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:25.639144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:25.639723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.639723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:25.639723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.639723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:25.645531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:25.645782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:25.645886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:25.646047 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:25.646472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:25.646723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:25.646914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.647428 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:25.647428 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.647428 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:25.647428 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:25.647428 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.647428 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:25.647428 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.647428 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.648354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.648656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:25.648972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.648972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:25.649331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:25.649662 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:25.650125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:25.650125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:25.650125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:30.364507 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:30.364904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:30.365032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:30.365672 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:30.391541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:30.391700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.392137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:30.392137 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.392137 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:30.392137 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:30.392137 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.392137 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:30.392137 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.392137 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.392790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.393032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:30.393250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.393250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:30.393496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:30.393871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.394247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:30.394247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.394247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:30.613309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:30.613573 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:30.613676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:30.614183 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:30.614324 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:30.614440 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:30.614545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.614969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:30.614969 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.614969 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:30.614969 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:30.614969 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.614969 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:30.614969 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.614969 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.615633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.615894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:30.616118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.616118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:30.616582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:30.616914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.617213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:30.617213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.617213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:30.618302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:30.618495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:30.618606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:30.618760 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:30.618996 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:30.619133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.619482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:30.619482 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.619482 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:30.619482 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:30.619482 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.619482 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:30.619482 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.619482 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:30.620010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:30.620283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.620283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:30.620283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.620283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:30.620283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:30.620283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.620283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:30.620283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.620283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.620582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:30.620815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:30.621045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.621045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:30.621045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.621045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:30.621256 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.621546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:30.621868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.621868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:30.621868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:30.621868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.621868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:30.621868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.621868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:30.634265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:30.634444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:30.634531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:30.634632 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:30.634878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:30.635062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.635393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:30.635393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.635393 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:30.635393 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:30.635393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.635393 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:30.635393 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.635393 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:30.635900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:30.636174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.636174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:30.636174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.636174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:30.636174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:30.636174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.636174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:30.636174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.636174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.636488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:30.636743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:30.636966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.636966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:30.636966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.636966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.637169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:30.637525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:30.637836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.637836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:30.637836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.637836 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:30.645855 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:30.645988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:30.646075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:30.646200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:30.646440 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:30.646609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:30.646752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.647090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:30.647090 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.647090 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:30.647090 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:30.647090 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.647090 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:30.647090 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.647090 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.647643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.647905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:30.648108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.648108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:30.648286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:30.648572 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:30.648905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:30.648905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:30.648905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:35.364940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:35.365992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:35.366095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:35.366709 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:35.391846 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:35.392020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.392368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:35.392368 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.392368 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:35.392368 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:35.392368 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.392368 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:35.392368 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.392368 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.392985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.393294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:35.393535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.393535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:35.393799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:35.394112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.394426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:35.394426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.394426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:35.613731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:35.614094 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:35.614205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:35.614915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:35.615149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:35.615292 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:35.615410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.615893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:35.615893 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.615893 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:35.615893 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:35.615893 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.615893 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:35.615893 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.615893 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.616591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.616840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:35.617038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.617038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:35.617307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:35.617588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:35.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.618200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:35.618555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:35.618771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:35.618854 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:35.618944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:35.619298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:35.619527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.619899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:35.619899 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.619899 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:35.619899 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:35.619899 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.619899 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:35.619899 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.619899 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:35.620502 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:35.620852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.620852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:35.620852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.620852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:35.620852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:35.620852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.620852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:35.620852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.620852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.621242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:35.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:35.621809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.622055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.622355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:35.634547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:35.634971 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:35.635191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:35.635360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:35.635903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:35.636108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.636722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:35.636722 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.636722 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:35.636722 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:35.636722 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.636722 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:35.636722 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.636722 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:35.637329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:35.637615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.637615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:35.637615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.637615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:35.637615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:35.637615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.637615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:35.637615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.637615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.637955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:35.638286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:35.638495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.638495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:35.638495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.638495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.638993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:35.639363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:35.639700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.639700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:35.639700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.639700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:35.646092 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:35.646415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:35.646501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:35.646659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:35.647049 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:35.647212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:35.647334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.647755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:35.647755 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.647755 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:35.647755 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:35.647755 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.647755 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:35.647755 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.647755 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.648458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.648695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:35.648892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.648892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:35.649160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:35.649442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:35.649752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:35.649752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:35.649752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:40.366122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:40.366528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:40.366528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:40.367046 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:40.392326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:40.392542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.393116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:40.393116 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.393116 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:40.393116 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:40.393116 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.393116 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:40.393116 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.393116 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.393841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.394087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:40.394281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.394281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:40.394742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:40.395440 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.395829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:40.395829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.395829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:40.614205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:40.614487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:40.614600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:40.615341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:40.615542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:40.615684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:40.615796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.616313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:40.616313 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.616313 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:40.616313 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:40.616313 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.616313 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:40.616313 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.616313 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.617012 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.617209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:40.617409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.617409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:40.617969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:40.618214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.618596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:40.618596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.618596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:40.618785 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:40.618948 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:40.619047 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:40.619142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:40.619443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:40.619598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.620034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:40.620034 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.620034 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:40.620034 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:40.620034 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.620034 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:40.620034 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.620034 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:40.620600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:40.620859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.620859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:40.620859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.620859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:40.620859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:40.620859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.620859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:40.620859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.620859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.621253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:40.621698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:40.622032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.622234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.622515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:40.635008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:40.635304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:40.635386 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:40.635502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:40.635867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:40.636079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.636499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:40.636499 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.636499 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:40.636499 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:40.636499 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.636499 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:40.636499 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.636499 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:40.637136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:40.637359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.637359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:40.637359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.637359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:40.637359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:40.637359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.637359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:40.637359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.637359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.637775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:40.638183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:40.638476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:40.638702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.638978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:40.646414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:40.646618 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:40.646730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:40.646910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:40.647213 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:40.647406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:40.647531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.647942 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:40.647942 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.647942 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:40.647942 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:40.647942 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.647942 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:40.647942 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.647942 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:40.649046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.649046 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:40.649359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:40.649602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:40.650000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:40.650000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:40.650000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:45.366587 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:45.367005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:45.367109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:45.367911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:45.395752 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:45.395995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.396637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:45.396637 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.396637 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:45.396637 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:45.396637 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.396637 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:45.396637 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.396637 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.397373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.397691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:45.397990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.397990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:45.398155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:45.398556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.398872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:45.398872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.398872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:45.614858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:45.615045 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:45.615194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:45.615836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:45.616034 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:45.616158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:45.616274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.616746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:45.616746 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.616746 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:45.616746 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:45.616746 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.616746 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:45.616746 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.616746 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.617456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.617688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:45.617920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.617920 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:45.618237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:45.618521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.618865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:45.618865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.618865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:45.619031 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:45.619219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:45.619299 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:45.619397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:45.619671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:45.619842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.620252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:45.620252 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.620252 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:45.620252 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:45.620252 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.620252 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:45.620252 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.620252 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:45.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:45.621251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.621251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:45.621251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.621251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:45.621251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:45.621251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.621251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:45.621251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.621251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.621509 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:45.621890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:45.622283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.622686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.623290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:45.635329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:45.635588 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:45.635710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:45.635833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:45.636124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:45.636294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.636699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:45.636699 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.636699 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:45.636699 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:45.636699 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.636699 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:45.636699 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.636699 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:45.637266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:45.637729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.637729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:45.637729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.637729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:45.637729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:45.637729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.637729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:45.637729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.637729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.638017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:45.638346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:45.638735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:45.639111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.639602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:45.646743 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:45.646940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:45.647031 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:45.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:45.647432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:45.647594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:45.647764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.648264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:45.648264 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.648264 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:45.648264 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:45.648264 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.648264 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:45.648264 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.648264 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.648986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.649218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:45.649449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.649449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:45.649669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:45.650043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:45.650347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:45.650347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:45.650347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:50.367049 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:50.367486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:50.367486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:50.368024 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:50.393250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:50.393371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.393831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:50.393831 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.393831 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:50.393831 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:50.393831 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.393831 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:50.393831 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.393831 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.394420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.394792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:50.395137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.395137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:50.395259 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:50.395718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.396020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:50.396020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.396020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:50.615068 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:50.615331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:50.615406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:50.616109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:50.616231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:50.616364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:50.616512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.617055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:50.617055 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.617055 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:50.617055 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:50.617055 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.617055 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:50.617055 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.617055 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.618566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.618838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:50.619480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:50.620203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.620691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:50.620691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:50.620691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.620691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:50.620691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.620691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:50.620691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:50.620691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:50.620691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:50.620691 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:50.620963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:50.620963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.620963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:50.620963 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.620963 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:50.620963 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:50.620963 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.620963 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:50.620963 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.620963 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:50.621269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:50.621701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.621701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:50.621701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.621701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:50.621701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:50.621701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.621701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:50.621701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.621701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.622151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:50.622505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:50.622916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.622916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:50.622916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.622916 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:50.623224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.623539 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:50.623857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.623857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:50.623857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:50.623857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.623857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:50.623857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.623857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:50.635665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:50.635898 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:50.636016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:50.636173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:50.636487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:50.636699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.637067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:50.637067 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.637067 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:50.637067 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:50.637067 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.637067 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:50.637067 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.637067 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:50.637566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:50.638008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.638008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:50.638008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.638008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:50.638008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:50.638008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.638008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:50.638008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.638008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:50.638520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:50.638906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:50.639265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.639805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:50.647111 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:50.647284 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:50.647408 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:50.647618 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:50.647913 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:50.648137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:50.648326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.648737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:50.648737 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.648737 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:50.648737 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:50.648737 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.648737 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:50.648737 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.648737 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.649295 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.649637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:50.649985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:50.650353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:50.650910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:50.650910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:50.650910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:50.650910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:50.650910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:50.650910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:55.367466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:55.367934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:55.367934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:55.368316 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:55.371868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=240 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:21:55.396377 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:55.396520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.397481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:55.397481 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.397481 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:55.397481 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:55.397481 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.397481 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:55.397481 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.397481 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.397707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.397929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:55.398148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.398148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:55.398552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:55.398825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.399147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:55.399147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.399147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:55.615594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:55.615758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:55.615851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:55.616446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:55.616605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:55.616744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:55.616881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.617325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:55.617325 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.617325 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:55.617325 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:55.617325 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.617325 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:55.617325 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.617325 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.617954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.618184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:55.618372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.618372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:55.618733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:55.619310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.619771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:55.619771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.619771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:55.619771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:55.619771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:55.619771 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:55.619925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:21:55.620140 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:55.620293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.620670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:55.620670 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.620670 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:55.620670 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:55.620670 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.620670 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:55.620670 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.620670 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:55.621154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:55.621543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.621543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:55.621543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.621543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:55.621543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:55.621543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.621543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:55.621543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.621543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.622015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:55.622281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:55.622496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.622736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.623034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:55.636085 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:55.636302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:55.636334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:55.636515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:55.636863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:55.637068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:55.637515 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.637515 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:55.637515 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:55.637515 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.637515 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:55.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.637515 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:21:55.638034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:21:55.638357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.638357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:21:55.638357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.638357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:21:55.638357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:21:55.638357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.638357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:21:55.638357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.638357 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.638890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:55.639185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:55.639618 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:55.640093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.640382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:21:55.647428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:21:55.648494 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:21:55.648633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:21:55.648824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:21:55.649150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:55.649361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:21:55.649569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.650129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:21:55.650129 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.650129 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:21:55.650129 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:21:55.650129 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.650129 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:21:55.650129 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.650129 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.650839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.651180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:21:55.651585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:21:55.652078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:21:55.652381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:21:55.652381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:21:55.652381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:21:55.652381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:21:55.652381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:21:55.652381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:00.367873 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:00.368387 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:00.368528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:00.369206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:00.402196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:00.402416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.403018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:00.403018 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.403018 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:00.403018 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:00.403018 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.403018 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:00.403018 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.403018 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.404275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:00.404684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:00.405135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.405771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:00.405771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:00.405771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.405771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:00.405771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.405771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:00.615957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:00.616242 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:00.616339 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:00.616904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:00.617104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:00.617272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:00.617407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.617945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:00.617945 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.617945 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:00.617945 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:00.617945 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.617945 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:00.617945 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.617945 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.618556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.618876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:00.619096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.619096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:00.619278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:00.619581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.620017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:00.620017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.620017 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:00.620017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:00.620017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:00.620017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:00.620153 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:00.620396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:00.620637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.621100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:00.621100 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.621100 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:00.621100 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:00.621100 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.621100 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:00.621100 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.621100 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:00.621574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:00.621944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.621944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:00.621944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.621944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:00.621944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:00.621944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.621944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:00.621944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.621944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.622146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:00.622411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:00.622636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.622636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:00.622636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.622636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:00.622851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.623137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:00.623450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.623450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:00.623450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:00.623450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.623450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:00.623450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.623450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:00.636379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:00.636597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:00.636739 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:00.636883 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:00.637188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:00.637412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.637785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:00.637785 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.637785 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:00.637785 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:00.637785 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.637785 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:00.637785 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.637785 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:00.638415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:00.638863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.638863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:00.638863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.638863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:00.638863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:00.638863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.638863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:00.638863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.638863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.639318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:00.639676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:00.640041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.640041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:00.640041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.640041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.640394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:00.640745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:00.641055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.641055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:00.641055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.641055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:00.648254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:00.648453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:00.648574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:00.648757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:00.649011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:00.649200 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:00.649325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.649682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:00.649682 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.649682 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:00.649682 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:00.649682 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.649682 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:00.649682 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.649682 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.650411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.650782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:00.651172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.651172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:00.651423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:00.651739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:00.652040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:00.652040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:00.652040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:05.368478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:05.368901 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:05.368901 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:05.369532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:05.398391 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:05.398521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.398978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:05.398978 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.398978 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:05.398978 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:05.398978 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.398978 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:05.398978 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.398978 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.399602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.399845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:05.400062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.400062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:05.400346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:05.400600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.400899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:05.400899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.400899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:05.616346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:05.616726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:05.616852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:05.617470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:05.617671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:05.617782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:05.617892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.618368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:05.618368 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.618368 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:05.618368 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:05.618368 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.618368 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:05.618368 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.618368 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.619116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.619325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:05.619598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.619598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:05.619843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:05.620113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.620488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:05.620488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.620488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:05.620488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:05.620488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:05.620488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:05.620596 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:05.620863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:05.621082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.621605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:05.621605 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.621605 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:05.621605 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:05.621605 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.621605 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:05.621605 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.621605 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:05.622177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:05.622423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.622423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:05.622423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.622423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:05.622423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:05.622423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.622423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:05.622423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.622423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.622764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:05.622951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:05.623171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.623171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:05.623171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.623171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:05.623606 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.624112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:05.624526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.624526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:05.624526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:05.624526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.624526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:05.624526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.624526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:05.636767 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:05.637053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:05.637199 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:05.637373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:05.637718 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:05.637913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.638319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:05.638319 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.638319 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:05.638319 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:05.638319 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.638319 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:05.638319 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.638319 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:05.638855 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:05.639128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.639128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:05.639128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.639128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:05.639128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:05.639128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.639128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:05.639128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.639128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.639462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:05.639669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:05.639894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.639894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:05.639894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.639894 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.640189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:05.640478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:05.641075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.641075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:05.641075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.641075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:05.648556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:05.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:05.648890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:05.649078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:05.649370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:05.649605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:05.649823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.650302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:05.650302 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.650302 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:05.650302 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:05.650302 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.650302 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:05.650302 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.650302 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.651065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.651275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:05.651537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.651537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:05.652045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:05.652339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:05.652657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:05.652657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:05.652657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:10.368958 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:10.371874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:10.371874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:10.371874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:10.398643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:10.398930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:10.400335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:10.401100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:10.401604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:10.402216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:10.402997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.402997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:10.402997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.402997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:10.402997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:10.402997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.402997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:10.402997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.402997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:10.616889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:10.617273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:10.617358 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:10.618129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:10.618416 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:10.618636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:10.618848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.619564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:10.619564 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.619564 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:10.619564 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:10.619564 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.619564 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:10.619564 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.619564 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.620294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.620613 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:10.620937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.620937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:10.621277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:10.622008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.622296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:10.622296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.622296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:10.622296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:10.622296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:10.622296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:10.622296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:10.622296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:10.622781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.623071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:10.623071 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.623071 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:10.623071 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:10.623071 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.623071 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:10.623071 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.623071 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:10.623551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:10.623939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.623939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:10.623939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.623939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:10.623939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:10.623939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.623939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:10.623939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.623939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.624141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:10.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:10.624970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.625191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.625499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:10.637171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:10.637506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:10.637610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:10.637744 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:10.638197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:10.638431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.639004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:10.639004 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.639004 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:10.639004 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:10.639004 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.639004 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:10.639004 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.639004 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:10.639803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:10.640105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.640105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:10.640105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.640105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:10.640105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:10.640105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.640105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:10.640105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.640105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.640477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:10.640720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:10.640932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.640932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:10.640932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.640932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.641346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:10.641932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:10.642237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.642237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:10.642237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.642237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:10.649047 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:10.649267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:10.649360 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:10.649517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:10.649930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:10.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:10.650306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.650799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:10.650799 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.650799 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:10.650799 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:10.650799 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.650799 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:10.650799 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.650799 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.651508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.651805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:10.652041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.652041 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:10.652321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:10.652867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:10.653176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:10.653176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:10.653176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:15.370103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:15.370571 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:15.370571 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:15.371231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:15.399334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:15.399434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.399869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:15.399869 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.399869 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:15.399869 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:15.399869 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.399869 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:15.399869 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.399869 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.400482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.400753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:15.401040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.401040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:15.401175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:15.401504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.401874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:15.401874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.401874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:15.617373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:15.617658 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:15.617752 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:15.618307 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:15.618437 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:15.618553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:15.618657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.619069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:15.619069 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.619069 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:15.619069 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:15.619069 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.619069 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:15.619069 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.619069 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.619764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.620047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:15.620241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.620241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:15.620437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:15.620766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.621080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:15.621080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.621080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:15.621694 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:15.621849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:15.621928 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:15.622024 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:15.622240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:15.622367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.622687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:15.622687 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.622687 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:15.622687 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:15.622687 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.622687 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:15.622687 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.622687 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:15.623154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:15.623529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.623529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:15.623529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.623529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:15.623529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:15.623529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.623529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:15.623529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.623529 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.623811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:15.624053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:15.624278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.624278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:15.624278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.624278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:15.624510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.624822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:15.625121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.625121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:15.625121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:15.625121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.625121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:15.625121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.625121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:15.637529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:15.637781 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:15.637884 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:15.638021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:15.638304 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:15.638495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.638907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:15.638907 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.638907 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:15.638907 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:15.638907 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.638907 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:15.638907 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.638907 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:15.639550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:15.639861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.639861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:15.639861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.639861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:15.639861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:15.639861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.639861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:15.639861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.639861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.640300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:15.640524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:15.640798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.640798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:15.640798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.640798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.641240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:15.641560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:15.641893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.641893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:15.641893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.641893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:15.649280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:15.649517 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:15.649567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:15.649748 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:15.650026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:15.650235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:15.650383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.650783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:15.650783 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.650783 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:15.650783 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:15.650783 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.650783 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:15.650783 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.650783 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.651459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.651706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:15.651941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.651941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:15.652300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:15.652655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:15.652966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:15.652966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:15.652966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:20.370590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:20.371217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:20.371217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:20.371655 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:20.400472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:20.400607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.401226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:20.401226 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.401226 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:20.401226 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:20.401226 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.401226 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:20.401226 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.401226 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.402000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.402278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:20.402483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.402483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:20.402735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:20.403059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.403389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:20.403389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.403389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:20.617874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:20.618355 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:20.618477 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:20.619282 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:20.619490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:20.619633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:20.619758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.620299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:20.620299 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.620299 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:20.620299 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:20.620299 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.620299 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:20.620299 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.620299 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.621325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.621568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:20.621782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.621782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:20.622389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:20.622921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.623454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:20.623454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.623454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:20.623454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:20.623454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:20.623454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:20.623454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:20.623454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:20.623847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.623998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:20.623998 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.623998 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:20.623998 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:20.623998 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.623998 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:20.623998 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.623998 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:20.624750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:20.625210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.625210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:20.625210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.625210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:20.625210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:20.625210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.625210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:20.625210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.625210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.625608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:20.625983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:20.626368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.626368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:20.626368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.626368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:20.626785 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.627085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:20.627430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.627430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:20.627430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:20.627430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.627430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:20.627430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.627430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:20.637868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:20.638139 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:20.638250 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:20.638361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:20.638652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:20.638860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.639277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:20.639277 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.639277 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:20.639277 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:20.639277 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.639277 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:20.639277 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.639277 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:20.639880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:20.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:20.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:20.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:20.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:20.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.640282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.640506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:20.640741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:20.640968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.640968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:20.640968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.640968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.641299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:20.641664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:20.641964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.641964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:20.641964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.641964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:20.649563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:20.649736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:20.649857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:20.650037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:20.650586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:20.650760 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:20.650902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.651540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:20.651540 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.651540 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:20.651540 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:20.651540 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.651540 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:20.651540 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.651540 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.652211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.652517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:20.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:20.652963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:20.653333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:20.653679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:20.653679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:20.653679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:25.371006 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:25.371496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:25.371539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:25.372328 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:25.401894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:25.402106 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:25.404871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:25.405320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:25.405671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:25.405992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:25.406478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.406478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:25.406478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.406478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:25.406478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:25.406478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.406478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:25.406478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.406478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:25.618486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:25.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:25.619582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:25.620702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:25.620702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:25.620702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:25.620702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.620881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:25.620881 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.620881 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:25.620881 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:25.620881 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.620881 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:25.620881 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.620881 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.623206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.623632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:25.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:25.624298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.624770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:25.624770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:25.624770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.624770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:25.624770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.624770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:25.624770 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:25.626217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:25.626217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:25.626217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:25.626427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:25.626845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.629827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:25.630420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.630853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.631200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:25.631579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:25.631986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:25.632499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.632499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:25.632499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.632499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:25.632499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:25.632499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.632499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:25.632499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.632499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:25.638194 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:25.638613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:25.638869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:25.639081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:25.639443 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:25.639632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:25.643255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.643684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.644033 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:25.644381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.644381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:25.645135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:25.645643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.645643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:25.645643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.645643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:25.649879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:25.650500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:25.650500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:25.650500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:25.650716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:25.650879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:25.650999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:25.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:25.653747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:25.654109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:25.654499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:25.655141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:25.655141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:25.655141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:25.655141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:25.655141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:25.655141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:25.655141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:25.655141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:25.655141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:25.655141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:30.371461 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:30.371962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:30.371962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:30.372378 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:30.398589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:30.398813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.399349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:30.399349 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.399349 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:30.399349 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:30.399349 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.399349 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:30.399349 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.399349 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.400051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.400321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:30.400513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.400513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:30.400780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:30.401316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.401821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:30.401821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.401821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:30.618929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:30.619117 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:30.619209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:30.619941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:30.620133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:30.620256 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:30.620363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.620839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:30.620839 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.620839 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:30.620839 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:30.620839 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.620839 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:30.620839 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.620839 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.621500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.621784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:30.621980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.621980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:30.622277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:30.622949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.623304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:30.623304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.623304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:30.625850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:30.626018 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:30.626103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:30.626190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:30.626479 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:30.626651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.627147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:30.627147 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.627147 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:30.627147 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:30.627147 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.627147 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:30.627147 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.627147 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:30.627740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:30.627998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.627998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:30.627998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.627998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:30.627998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:30.627998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.627998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:30.627998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.627998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.628275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:30.628716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:30.629075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.629297 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.629563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:30.638580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:30.638872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:30.638961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:30.639060 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:30.639523 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:30.639763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.640183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:30.640183 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.640183 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:30.640183 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:30.640183 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.640183 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:30.640183 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.640183 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:30.640795 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:30.641058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.641058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:30.641058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.641058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:30.641058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:30.641058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.641058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:30.641058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.641058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.641631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:30.641844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:30.642069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.642069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:30.642069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.642069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.642703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:30.642980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:30.643292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.643292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:30.643292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.643292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:30.650186 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:30.650395 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:30.650480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:30.650628 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:30.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:30.651101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:30.651224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.651608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:30.651608 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.651608 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:30.651608 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:30.651608 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.651608 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:30.651608 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.651608 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.652286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.652486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:30.652716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.652716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:30.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:30.653250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:30.653516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:30.653516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:30.653516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:35.371842 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:35.372470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:35.372470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:35.373298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:35.402481 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:35.402733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.403334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:35.403334 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.403334 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:35.403334 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:35.403334 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.403334 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:35.403334 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.403334 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.404212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.404485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:35.404723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.404723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:35.405080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:35.405361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.405701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:35.405701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.405701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:35.619248 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:35.619482 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:35.619556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:35.620264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:35.620501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:35.620662 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:35.620788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.621260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:35.621260 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.621260 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:35.621260 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:35.621260 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.621260 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:35.621260 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.621260 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.622103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.622348 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:35.622568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.622568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:35.623001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:35.623377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.623709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:35.623709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.623709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:35.626065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:35.626241 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:35.626317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:35.626450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:35.626817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:35.627051 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:35.627546 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.627546 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:35.627546 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:35.627546 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.627546 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:35.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.627546 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:35.628132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:35.628432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.628432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:35.628432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.628432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:35.628432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:35.628432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.628432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:35.628432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.628432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.628783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:35.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:35.629655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.629888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.630176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:35.638902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:35.639079 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:35.639180 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:35.639249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:35.639747 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:35.640061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.640583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:35.640583 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.640583 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:35.640583 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:35.640583 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.640583 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:35.640583 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.640583 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:35.641120 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:35.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:35.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:35.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:35.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:35.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.641750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:35.641971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:35.642173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.642173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:35.642173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.642173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:35.643113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:35.643560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.643560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:35.643560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.643560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:35.650526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:35.650802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:35.650934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:35.651118 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:35.651498 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:35.651750 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:35.651944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.652562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:35.652562 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.652562 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:35.652562 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:35.652562 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.652562 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:35.652562 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.652562 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.653463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.653707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:35.653908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.653908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:35.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:35.654779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:35.655225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:35.655225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:35.655225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:40.372332 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:40.372788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:40.372829 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:40.373417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:40.399741 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:40.399964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.400549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:40.400549 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.400549 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:40.400549 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:40.400549 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.400549 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:40.400549 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.400549 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.401303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.401535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:40.401779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.401779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:40.402004 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:40.402306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.402617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:40.402617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.402617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:40.619641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:40.619979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:40.620067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:40.620901 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:40.621088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:40.621238 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:40.621366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.621898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:40.621898 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.621898 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:40.621898 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:40.621898 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.621898 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:40.621898 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.621898 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.622648 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.622890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:40.623124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.623124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:40.623269 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:40.623557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.623924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:40.623924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.623924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:40.626341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:40.626533 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:40.626646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:40.626772 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:40.627098 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:40.627284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.627757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:40.627757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.627757 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:40.627757 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:40.627757 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.627757 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:40.627757 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.627757 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:40.628253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:40.628523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.628523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:40.628523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.628523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:40.628523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:40.628523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.628523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:40.628523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.628523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.628784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:40.628990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:40.629292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.629292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:40.629292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.629292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:40.629420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.629713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:40.630029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.630029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:40.630029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:40.630029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.630029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:40.630029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.630029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:40.639193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:40.639414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:40.639502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:40.639640 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:40.639892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:40.640107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.640412 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:40.640412 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.640412 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:40.640412 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:40.640412 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.640412 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:40.640412 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.640412 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:40.640882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:40.641248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.641248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:40.641248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.641248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:40.641248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:40.641248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.641248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:40.641248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.641248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:40.641673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:40.641929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.641929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:40.641929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.641929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.642083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:40.642410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:40.642742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.642742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:40.642742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.642742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:40.650882 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:40.651095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:40.651186 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:40.651315 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:40.651529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:40.651766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:40.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.652314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:40.652314 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.652314 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:40.652314 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:40.652314 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.652314 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:40.652314 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.652314 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.652860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.653089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:40.653326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.653326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:40.653574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:40.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:40.654187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:40.654187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:40.654187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:45.372813 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:45.373336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:45.373336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:45.373934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:45.415678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:45.415906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.416460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:45.416460 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.416460 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:45.416460 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:45.416460 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.416460 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:45.416460 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.416460 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.417160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.417611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:45.417988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.417988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:45.418342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:45.418866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.419393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:45.419393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.419393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:45.620462 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:45.620686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:45.620762 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:45.621445 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:45.621589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:45.621716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:45.621876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.622293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:45.622293 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.622293 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:45.622293 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:45.622293 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.622293 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:45.622293 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.622293 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.623028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.623394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:45.623764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.623764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:45.623878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:45.624309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.624837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:45.624837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.624837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:45.626574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:45.626768 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:45.626853 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:45.626952 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:45.627228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:45.627384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.627776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:45.627776 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.627776 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:45.627776 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:45.627776 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.627776 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:45.627776 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.627776 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:45.628347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:45.628883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.628883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:45.628883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.628883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:45.628883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:45.628883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.628883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:45.628883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.628883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.629152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:45.629515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:45.629949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.630311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.630847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:45.639570 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:45.639813 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:45.639898 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:45.640015 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:45.640302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:45.640532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.641039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:45.641039 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.641039 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:45.641039 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:45.641039 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.641039 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:45.641039 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.641039 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:45.641756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:45.642062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.642062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:45.642062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.642062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:45.642062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:45.642062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.642062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:45.642062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.642062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.642541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:45.642767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:45.643018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.643018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:45.643018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.643018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.643513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:45.643879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:45.644287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.644287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:45.644287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.644287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:45.651265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:45.651528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:45.651648 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:45.651819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:45.652110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:45.652341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:45.652483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.652924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:45.652924 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.652924 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:45.652924 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:45.652924 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.652924 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:45.652924 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.652924 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.653603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.653946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:45.654215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.654215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:45.654564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:45.654934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:45.655326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:45.655326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:45.655326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:50.373252 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:50.373637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:50.373725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:50.374271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:50.402084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:50.402233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.402636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:50.402636 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.402636 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:50.402636 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:50.402636 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.402636 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:50.402636 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.402636 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.403289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.403517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:50.403773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.403773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:50.403986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:50.404236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.404540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:50.404540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.404540 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:50.620896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:50.621145 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:50.621237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:50.621805 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:50.621963 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:50.622067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:50.622167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.622570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:50.622570 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.622570 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:50.622570 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:50.622570 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.622570 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:50.622570 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.622570 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.623202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.623578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:50.624019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:50.624449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.624998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:50.624998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:50.624998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.624998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:50.624998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.624998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:50.626836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:50.626993 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:50.627074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:50.627161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:50.627377 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:50.627517 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.627864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:50.627864 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.627864 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:50.627864 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:50.627864 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.627864 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:50.627864 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.627864 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:50.628362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:50.628645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.628645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:50.628645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.628645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:50.628645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:50.628645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.628645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:50.628645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.628645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.628988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:50.629369 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:50.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:50.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.629778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:50.629899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.630430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:50.630955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.630955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:50.630955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:50.630955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.630955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:50.630955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.630955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:50.639934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:50.640136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:50.640223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:50.640321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:50.640586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:50.640809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.641191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:50.641191 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.641191 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:50.641191 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:50.641191 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.641191 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:50.641191 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.641191 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:50.641744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:50.642148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.642148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:50.642148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.642148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:50.642148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:50.642148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.642148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:50.642148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.642148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.642374 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:50.642694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:50.643044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.643044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:50.643044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.643044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.643211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:50.643665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:50.644122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.644122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:50.644122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.644122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:50.651559 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:50.651794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:50.651879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:50.652032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:50.652466 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:50.652723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:50.652872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.653352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:50.653352 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.653352 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:50.653352 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:50.653352 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.653352 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:50.653352 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.653352 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.654001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.654310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:50.654634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.654634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:50.654708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:50.655104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:50.655553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:50.655553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:50.655553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:55.373677 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:55.374212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:55.374212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:55.374554 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:55.375757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=239 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:22:55.401461 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:55.401823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.402501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:55.402501 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.402501 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:55.402501 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:55.402501 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.402501 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:55.402501 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.402501 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.403477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.403849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:55.404198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.404198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:55.404551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:55.404877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.405173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:55.405173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.405173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:55.621331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:55.621569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:55.621715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:55.622492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:55.622753 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:55.622936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:55.623122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.623665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:55.623665 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.623665 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:55.623665 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:55.623665 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.623665 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:55.623665 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.623665 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.624497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.624768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:55.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.625180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:55.625706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:55.626174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.626710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:55.626710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.626710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:55.627086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:55.627265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:55.627348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:55.627471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:22:55.627834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:55.628019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.628487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:55.628487 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.628487 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:55.628487 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:55.628487 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.628487 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:55.628487 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.628487 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:55.629196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:55.629458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.629458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:55.629458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.629458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:55.629458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:55.629458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.629458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:55.629458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.629458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.629896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:55.630122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:55.630341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.630341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:55.630341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.630341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:55.630550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.630896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:55.631216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.631216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:55.631216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:55.631216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.631216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:55.631216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.631216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:55.640223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:55.640475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:55.640573 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:55.641003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:55.641233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.641737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:55.641737 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.641737 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:55.641737 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:55.641737 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.641737 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:55.641737 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.641737 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:22:55.642278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:22:55.642569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.642569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:22:55.642569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.642569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:22:55.642569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:22:55.642569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.642569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:22:55.642569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.642569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.642889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:55.643087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:55.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:55.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.643567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:55.643888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:55.644219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.644219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:55.644219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.644219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:55.651863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:55.652043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:55.652141 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:55.652282 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:22:55.652696 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:55.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:55.653114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.653590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:55.653590 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.653590 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:55.653590 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:55.653590 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.653590 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:55.653590 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.653590 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.654152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.654388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:55.654594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.654594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:22:55.654782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:22:55.655082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:55.655389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:22:55.655389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:55.655389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:22:57.816574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:22:57.817084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:22:57.817084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:22:57.823100 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 27 16:22:57.823257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Number of inconsistencies found at create/update: networks=5, routers=2, router_ports=2 {{(pid=92285) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 27 16:22:57.823396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Number of inconsistencies found at delete: router_ports=5 {{(pid=92285) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 27 16:22:57.823534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource e2510e0f-28d1-484d-8605-6e6fbc1e01e2 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:22:57.848207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e2510e0f-28d1-484d-8605-6e6fbc1e01e2, resource={'id': 'e2510e0f-28d1-484d-8605-6e6fbc1e01e2', 'name': 'private', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2524c773-2910-41f1-b8dd-03d9ac809fd5', '71e0c14f-375e-43c9-bf49-7597bd311ee9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T15:57:42Z', 'updated_at': '2026-05-27T15:57:44Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 8519}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.848365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e2510e0f-28d1-484d-8605-6e6fbc1e01e2, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.848568 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-96ddfe89-0ab3-4b19-b3ee-13a966e7d5c2, external_ids_update=None, columns={'tag': 8519}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.848675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.849113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:57.849113 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:57.849113 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:57.849113 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:57.849113 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:57.849113 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:57.849113 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.849113 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource e2510e0f-28d1-484d-8605-6e6fbc1e01e2 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:57.850014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:57.850255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:57.850486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:57.850486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.850486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:57.850486 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 1a1659c0-4943-4985-b3f1-75f0c26af59f (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:22:57.874113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, resource={'id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'name': 'public', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['156facd3-ecc0-4e78-8b24-b1c2e5631a78', '3c538124-e9fa-4f89-905c-f13de628c91d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-27T15:57:46Z', 'updated_at': '2026-05-27T15:57:53Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.874295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.874614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1d74a028-7065-4269-8c77-c99e0ea79147, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.874832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.875320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:57.875320 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:57.875320 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:57.875320 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:57.875320 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:57.875320 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:57.875320 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.875320 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 1a1659c0-4943-4985-b3f1-75f0c26af59f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:57.876234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:57.876430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:57.876690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:57.876690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.876690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:57.876690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 28588915-92a2-4ece-bd6f-c59cf1c236ce (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:22:57.899000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28588915-92a2-4ece-bd6f-c59cf1c236ce, resource={'id': '28588915-92a2-4ece-bd6f-c59cf1c236ce', 'name': 'shared', 'tenant_id': 'cf4659531f1043f08cb821c173ac8cf2', 'project_id': 'cf4659531f1043f08cb821c173ac8cf2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['73ef14d1-56af-457d-a4cc-cd982d0f7719'], 'standard_attr_id': 26, 'shared': True, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T15:58:53Z', 'updated_at': '2026-05-27T15:58:54Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 65356}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.899169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28588915-92a2-4ece-bd6f-c59cf1c236ce, col_values=(('external_ids', {'neutron:network_name': 'shared', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.899428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-854ee673-4c8a-4fe6-bb8f-35261288982d, external_ids_update=None, columns={'tag': 65356}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.899546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.900014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:57.900014 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:57.900014 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:57.900014 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:57.900014 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:57.900014 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:57.900014 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.900014 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 28588915-92a2-4ece-bd6f-c59cf1c236ce (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:57.900834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:57.901202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:57.901417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:57.901417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.901417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:57.901417 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource ec56e636-ffbb-4d63-87d1-b6b1dc7be102 (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:22:57.924354 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-ec56e636-ffbb-4d63-87d1-b6b1dc7be102, resource={'id': 'ec56e636-ffbb-4d63-87d1-b6b1dc7be102', 'name': 'tempest-TestVolumeBootPattern-1623358333-network', 'tenant_id': '1d9d90a978294926918bf1013e804d9b', 'project_id': '1d9d90a978294926918bf1013e804d9b', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['821ea963-808e-4b5e-a790-198868ae4a46'], 'standard_attr_id': 1721, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:16:10Z', 'updated_at': '2026-05-27T16:16:11Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 35046}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.924536 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-ec56e636-ffbb-4d63-87d1-b6b1dc7be102, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestVolumeBootPattern-1623358333-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.924790 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3a7bbffa-67a2-49bc-a12e-56c59ede0c97, external_ids_update=None, columns={'tag': 35046}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.924879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.925372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:57.925372 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:57.925372 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:57.925372 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:57.925372 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:57.925372 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:57.925372 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.925372 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource ec56e636-ffbb-4d63-87d1-b6b1dc7be102 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:57.926126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:57.926313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:57.926566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:57.926566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.926566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:57.926566 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 60e7b876-c3cc-4dbc-8d09-ec2fe5b5917a (type: networks) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:22:57.949647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-60e7b876-c3cc-4dbc-8d09-ec2fe5b5917a, resource={'id': '60e7b876-c3cc-4dbc-8d09-ec2fe5b5917a', 'name': 'tempest-TestVolumeBootPattern-1412527984-network', 'tenant_id': '514502f614654a7ea82df88f84bee31a', 'project_id': '514502f614654a7ea82df88f84bee31a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['a5367ee9-c0cf-472f-9ab0-6e77a12b5514'], 'standard_attr_id': 1744, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-27T16:16:17Z', 'updated_at': '2026-05-27T16:16:17Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 63684}, resource_type=networks, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.949845 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-60e7b876-c3cc-4dbc-8d09-ec2fe5b5917a, col_values=(('external_ids', {'neutron:network_name': 'tempest-TestVolumeBootPattern-1412527984-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.950098 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b8382ca-0035-47c2-9348-56afe07334c2, external_ids_update=None, columns={'tag': 63684}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:57.950185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.950600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:57.950600 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:57.950600 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:57.950600 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:57.950600 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:57.950600 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:57.950600 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.950600 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 60e7b876-c3cc-4dbc-8d09-ec2fe5b5917a (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2335, in update_network
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:57.951453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:57.951799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:57.952217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:57.952217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:57.952217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:57.952217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource dbce49bb-17ac-40db-91aa-b0d13e58bb7b (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:22:58.008784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, resource={'id': 'dbce49bb-17ac-40db-91aa-b0d13e58bb7b', 'name': 'tempest-TestVolumeBootPattern-1623358333-router', 'project_id': '1d9d90a978294926918bf1013e804d9b', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.109'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::bd'}], 'enable_snat': True}, 'gw_port_id': '513a2a7a-94d2-4cba-a15e-1f67ba36637d', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.109', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::bd', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:16:11Z', 'updated_at': '2026-05-27T16:16:14Z', 'revision_number': 3, 'tenant_id': '1d9d90a978294926918bf1013e804d9b'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.008972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, columns={'external_ids': {'neutron:router_name': 'tempest-TestVolumeBootPattern-1623358333-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.009090 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "513a2a7a-94d2-4cba-a15e-1f67ba36637d"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.009197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "513a2a7a-94d2-4cba-a15e-1f67ba36637d"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.009287 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.009719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:58.009719 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:58.009719 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:58.009719 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:58.009719 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:58.009719 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:58.009719 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.009719 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:58.009863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router dbce49bb-17ac-40db-91aa-b0d13e58bb7b. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource dbce49bb-17ac-40db-91aa-b0d13e58bb7b (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:22:58.011254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:58.011605 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:58.012027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:58.012027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:58.012027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:58.012027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:58.012027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:58.012027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:58.012027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:58.012027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.012027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:58.012027 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource cb0982dc-f84e-421b-ae33-411ef007397f (type: routers) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:22:58.085268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, resource={'id': 'cb0982dc-f84e-421b-ae33-411ef007397f', 'name': 'tempest-TestVolumeBootPattern-1412527984-router', 'project_id': '514502f614654a7ea82df88f84bee31a', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.232'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::1de'}], 'enable_snat': True}, 'gw_port_id': '03060c4d-30ec-4bc1-96d0-168d95e31a89', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'external_fixed_ips': [{'ip_address': '172.24.5.232', 'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d'}, {'ip_address': '2001:db8::1de', 'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-27T16:16:18Z', 'updated_at': '2026-05-27T16:16:20Z', 'revision_number': 3, 'tenant_id': '514502f614654a7ea82df88f84bee31a'}, resource_type=routers, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.085422 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, columns={'external_ids': {'neutron:router_name': 'tempest-TestVolumeBootPattern-1412527984-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.085565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "03060c4d-30ec-4bc1-96d0-168d95e31a89"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.085689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "03060c4d-30ec-4bc1-96d0-168d95e31a89"')], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.085776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.086209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:58.086209 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:58.086209 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:58.086209 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:58.086209 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:58.086209 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:58.086209 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.086209 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:58.086457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Unable to update router cb0982dc-f84e-421b-ae33-411ef007397f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource cb0982dc-f84e-421b-ae33-411ef007397f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1638, in update_router
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 27 16:22:58.087250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1572, in update_router
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:58.087463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:58.087726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:58.087726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:58.087726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:58.087726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:58.087726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:58.087726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:58.087726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:58.087726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.087726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:58.087726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 513a2a7a-94d2-4cba-a15e-1f67ba36637d (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:22:58.170191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-513a2a7a-94d2-4cba-a15e-1f67ba36637d, resource={'id': '513a2a7a-94d2-4cba-a15e-1f67ba36637d', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:d9:3c:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dbce49bb-17ac-40db-91aa-b0d13e58bb7b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1726, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.109'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::bd'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:16:12Z', 'updated_at': '2026-05-27T16:16:15Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.170460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-513a2a7a-94d2-4cba-a15e-1f67ba36637d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.109/24', '2001:db8::bd/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.170590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=513a2a7a-94d2-4cba-a15e-1f67ba36637d, lrouter_port=lrp-513a2a7a-94d2-4cba-a15e-1f67ba36637d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.170773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.171217 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:58.171217 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:58.171217 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:58.171217 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:58.171217 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:58.171217 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:58.171217 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.171217 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 513a2a7a-94d2-4cba-a15e-1f67ba36637d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:58.171930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:58.172198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:58.172454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:58.172454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.172454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:58.172454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 03060c4d-30ec-4bc1-96d0-168d95e31a89 (type: router_ports) at create/update {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 27 16:22:58.251817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-03060c4d-30ec-4bc1-96d0-168d95e31a89, resource={'id': '03060c4d-30ec-4bc1-96d0-168d95e31a89', 'name': '', 'network_id': '1a1659c0-4943-4985-b3f1-75f0c26af59f', 'project_id': '', 'mac_address': 'fa:16:3e:c6:36:b9', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cb0982dc-f84e-421b-ae33-411ef007397f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1749, 'fixed_ips': [{'subnet_id': '3c538124-e9fa-4f89-905c-f13de628c91d', 'ip_address': '172.24.5.232'}, {'subnet_id': '156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'ip_address': '2001:db8::1de'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000001523', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-27T16:16:18Z', 'updated_at': '2026-05-27T16:16:22Z', 'revision_number': 7, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.252051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-03060c4d-30ec-4bc1-96d0-168d95e31a89, columns={'external_ids': {'neutron:revision_number': '7', 'neutron:subnet_ids': '3c538124-e9fa-4f89-905c-f13de628c91d 156facd3-ecc0-4e78-8b24-b1c2e5631a78', 'neutron:network_name': 'neutron-1a1659c0-4943-4985-b3f1-75f0c26af59f', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-cb0982dc-f84e-421b-ae33-411ef007397f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::1de/64', '172.24.5.232/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.252228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=03060c4d-30ec-4bc1-96d0-168d95e31a89, lrouter_port=lrp-03060c4d-30ec-4bc1-96d0-168d95e31a89, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:22:58.252386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.252816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:22:58.252816 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:58.252816 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:22:58.252816 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:22:58.252816 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:58.252816 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:22:58.252816 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.252816 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Failed to fix resource 03060c4d-30ec-4bc1-96d0-168d95e31a89 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1979, in update_router_port
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:58.253490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:22:58.253727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:22:58.254026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:22:58.254026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:22:58.254026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:22:58.254026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 2ed0f212-2258-4166-a41a-3accb5e7ac55 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:22:58.254026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(2ed0f212-2258-4166-a41a-3accb5e7ac55) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:22:58.258177 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 41077fdf-a196-47b3-8ac4-e96813086e9a (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:22:58.258341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(41077fdf-a196-47b3-8ac4-e96813086e9a) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:22:58.262480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource 7d877385-7424-4d92-9669-65a6b38e2aa9 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:22:58.262635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(7d877385-7424-4d92-9669-65a6b38e2aa9) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:22:58.266397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource ec7fd787-d162-4873-8a27-4972b6b40184 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:22:58.266564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(ec7fd787-d162-4873-8a27-4972b6b40184) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:22:58.270326 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Maintenance task: Fixing resource f94a859c-f862-4b4c-951e-badd039a54d3 (type: router_ports) at delete {{(pid=92285) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:471}}
May 27 16:22:58.270491 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.db.ovn_revision_numbers_db [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] delete_revision(f94a859c-f862-4b4c-951e-badd039a54d3) {{(pid=92285) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}}
May 27 16:22:58.273703 np0000001523 neutron-ovn-maintenance-worker[92285]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_for_inconsistencies finished in 0.457 seconds
May 27 16:23:00.374070 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:00.374545 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:00.374545 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:00.374987 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:00.401310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:00.401679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.402275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:00.402275 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.402275 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:00.402275 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:00.402275 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.402275 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:00.402275 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.402275 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.403035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.403496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:00.403805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:00.404296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.404998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:00.404998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:00.404998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.404998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:00.404998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.404998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:00.621810 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:00.622106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:00.622198 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:00.623057 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:00.623242 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:00.623382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:00.623535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.624079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:00.624079 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.624079 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:00.624079 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:00.624079 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.624079 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:00.624079 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.624079 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.624696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.624950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:00.625194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.625194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:00.625431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:00.625709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.626050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:00.626050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.626050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:00.627355 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:00.627567 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:00.627679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:00.627806 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:00.628118 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:00.628313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.628825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:00.628825 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.628825 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:00.628825 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:00.628825 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.628825 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:00.628825 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.628825 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:00.629384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:00.629692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.629692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:00.629692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.629692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:00.629692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:00.629692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.629692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:00.629692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.629692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.629968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:00.630198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:00.630461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.630461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:00.630461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.630461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:00.630667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.630964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:00.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:00.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:00.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:00.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:00.640499 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:00.640728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:00.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:00.640908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:00.641174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:00.641334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.641674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:00.641674 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.641674 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:00.641674 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:00.641674 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.641674 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:00.641674 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.641674 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:00.642182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:00.642494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.642494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:00.642494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.642494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:00.642494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:00.642494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.642494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:00.642494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.642494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.642805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:00.643163 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:00.643541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:00.643960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.644485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:00.652160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:00.652382 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:00.652469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:00.652616 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:00.652982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:00.653193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:00.653315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.653716 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:00.653716 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.653716 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:00.653716 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:00.653716 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.653716 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:00.653716 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.653716 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.654400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.654756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:00.655107 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:00.655505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:00.656008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:00.656008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:00.656008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:00.656008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:00.656008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:00.656008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:05.374582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:05.374991 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:05.375120 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:05.375998 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:05.407889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:05.408081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.408512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:05.408512 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.408512 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:05.408512 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:05.408512 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.408512 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:05.408512 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.408512 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.409395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.409743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:05.409960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.409960 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:05.410421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:05.410769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.411085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:05.411085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.411085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:05.622275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:05.623467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:05.623467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:05.624163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:05.624403 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:05.624535 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:05.624732 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.625423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:05.625423 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.625423 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:05.625423 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:05.625423 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.625423 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:05.625423 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.625423 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.626429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.626792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:05.627037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.627037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:05.627660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:05.627971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.628270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:05.628270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.628270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:05.628270 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:05.628454 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:05.628654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:05.628814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:05.629604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:05.629790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.630393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:05.630393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.630393 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:05.630393 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:05.630393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.630393 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:05.630393 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.630393 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:05.630996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:05.631311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.631311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:05.631311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.631311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:05.631311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:05.631311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.631311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:05.631311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.631311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.631962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:05.632209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:05.632474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.632474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:05.632474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.632474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:05.633159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.633454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:05.633771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.633771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:05.633771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:05.633771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.633771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:05.633771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.633771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:05.640831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:05.642268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:05.642268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:05.642268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:05.642268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:05.642268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.643393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:05.643393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.643393 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:05.643393 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:05.643393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.643393 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:05.643393 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.643393 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:05.644541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:05.644981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.644981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:05.644981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.644981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:05.644981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:05.644981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.644981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:05.644981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.644981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.645243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:05.645601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:05.645995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.645995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:05.645995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.645995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.646250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:05.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:05.646932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.646932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:05.646932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.646932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:05.652432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:05.652642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:05.652732 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:05.652869 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:05.653330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:05.653561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:05.653746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.654289 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:05.654289 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.654289 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:05.654289 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:05.654289 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.654289 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:05.654289 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.654289 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.655034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.655358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:05.655727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.655727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:05.655846 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:05.656126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:05.656439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:05.656439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:05.656439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:10.375038 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:10.375580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:10.375580 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:10.375980 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:10.400670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:10.400912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:10.401473 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.401473 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:10.401473 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:10.401473 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.401473 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:10.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.401473 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.402081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.402298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:10.402665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.402665 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:10.402775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:10.403067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.403365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:10.403365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.403365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:10.623088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:10.623372 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:10.623547 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:10.624265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:10.624431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:10.624561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:10.624721 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.625174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:10.625174 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.625174 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:10.625174 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:10.625174 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.625174 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:10.625174 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.625174 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.626183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.626523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:10.626839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.626839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:10.627306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:10.627758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.628200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:10.628200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.628200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:10.628472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:10.628731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:10.628856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:10.629050 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:10.629367 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:10.629582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.630096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:10.630096 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.630096 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:10.630096 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:10.630096 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.630096 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:10.630096 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.630096 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:10.630907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:10.631266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.631266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:10.631266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.631266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:10.631266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:10.631266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.631266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:10.631266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.631266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.631806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:10.632157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:10.632463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.632463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:10.632463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.632463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:10.632910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.633243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:10.633617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.633617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:10.633617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:10.633617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.633617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:10.633617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.633617 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:10.641275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:10.641581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:10.641736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:10.641891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:10.642225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:10.642462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.643014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:10.643014 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.643014 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:10.643014 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:10.643014 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.643014 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:10.643014 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.643014 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:10.643852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:10.644168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.644168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:10.644168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.644168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:10.644168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:10.644168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.644168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:10.644168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.644168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:10.645031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:10.645246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.645246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:10.645246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.645246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.645932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:10.646264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:10.646563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.646563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:10.646563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.646563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:10.652865 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:10.653123 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:10.653251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:10.653453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:10.654020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:10.654217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:10.654353 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.654923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:10.654923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.654923 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:10.654923 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:10.654923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.654923 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:10.654923 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.654923 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.655696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.656029 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:10.656225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.656225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:10.656438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:10.656777 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:10.657116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:10.657116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:10.657116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:15.375985 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:15.376509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:15.376509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:15.377020 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:15.406914 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:15.407176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.407890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:15.407890 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.407890 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:15.407890 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:15.407890 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.407890 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:15.407890 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.407890 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.408787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.409328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:15.409796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:15.410307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.410954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:15.410954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:15.410954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.410954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:15.410954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.410954 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:15.623468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:15.623861 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:15.623949 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:15.624766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:15.625078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:15.625260 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:15.625438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.625982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:15.625982 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.625982 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:15.625982 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:15.625982 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.625982 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:15.625982 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.625982 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.626763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.626987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:15.627199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.627199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:15.627487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:15.628140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.628515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:15.628515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.628515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:15.628812 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:15.628972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:15.629073 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:15.629165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:15.629470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:15.629654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.630080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:15.630080 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.630080 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:15.630080 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:15.630080 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.630080 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:15.630080 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.630080 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:15.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:15.630888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.630888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:15.630888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.630888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:15.630888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:15.630888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.630888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:15.630888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.630888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.631338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:15.631726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:15.631940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.632166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.632449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:15.641606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:15.641834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:15.641918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:15.642043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:15.642421 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:15.642725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.643279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:15.643279 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.643279 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:15.643279 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:15.643279 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.643279 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:15.643279 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.643279 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:15.643831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:15.644090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.644090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:15.644090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.644090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:15.644090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:15.644090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.644090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:15.644090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.644090 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.644550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:15.645001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:15.645235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:15.645444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.645740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:15.653193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:15.653394 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:15.653479 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:15.653605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:15.653911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:15.654080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:15.654203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.654579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:15.654579 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.654579 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:15.654579 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:15.654579 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.654579 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:15.654579 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.654579 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.655184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.655383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:15.655570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.655570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:15.655917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:15.656514 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:15.656896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:15.656896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:15.656896 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:20.376441 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:20.376896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:20.376896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:20.377474 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:20.402262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:20.402550 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.403185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:20.403185 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.403185 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:20.403185 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:20.403185 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.403185 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:20.403185 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.403185 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.403900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.404099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:20.404321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.404321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:20.404619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:20.404891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.405172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:20.405172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.405172 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:20.623906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:20.624249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:20.624396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:20.625161 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:20.625331 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:20.625489 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:20.625603 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.626173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:20.626173 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.626173 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:20.626173 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:20.626173 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.626173 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:20.626173 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.626173 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.627192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.627503 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:20.627865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.627865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:20.628329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:20.628705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.629084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:20.629084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.629084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:20.629084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:20.629302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:20.629437 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:20.629573 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:20.629988 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:20.630142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.630582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:20.630582 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.630582 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:20.630582 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:20.630582 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.630582 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:20.630582 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.630582 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:20.631480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:20.631834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.631834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:20.631834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.631834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:20.631834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:20.631834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.631834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:20.631834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.631834 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.632334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:20.632592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:20.632812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.632812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:20.632812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.632812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:20.633493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.633799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:20.634125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.634125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:20.634125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:20.634125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.634125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:20.634125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.634125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:20.642016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:20.642319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:20.642470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:20.642634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:20.643028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:20.643232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.643739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:20.643739 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.643739 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:20.643739 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:20.643739 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.643739 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:20.643739 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.643739 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:20.644288 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:20.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:20.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:20.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:20.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:20.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.644995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:20.645530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:20.646049 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:20.646278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.646547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:20.653532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:20.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:20.653972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:20.654165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:20.654487 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:20.654692 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:20.654861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.655281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:20.655281 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.655281 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:20.655281 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:20.655281 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.655281 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:20.655281 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.655281 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.656066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.656495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:20.656833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:20.657220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:20.657694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:20.657694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:20.657694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:20.657694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:20.657694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:20.657694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:25.376818 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:25.377302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:25.377341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:25.378011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:25.405804 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:25.406055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.406591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:25.406591 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.406591 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:25.406591 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:25.406591 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.406591 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:25.406591 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.406591 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.407328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.407558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:25.407805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.407805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:25.408095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:25.408655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.408949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:25.408949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.408949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:25.624313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:25.624760 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:25.624894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:25.625641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:25.625872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:25.626007 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:25.626165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.626680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:25.626680 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.626680 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:25.626680 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:25.626680 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.626680 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:25.626680 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.626680 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.627432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.627659 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:25.627878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.627878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:25.628189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:25.628445 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.628776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:25.628776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.628776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:25.629343 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:25.629601 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:25.629702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:25.629800 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:25.630087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:25.630253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.630670 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:25.630670 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.630670 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:25.630670 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:25.630670 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.630670 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:25.630670 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.630670 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:25.631204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:25.631497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.631497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:25.631497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.631497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:25.631497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:25.631497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.631497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:25.631497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.631497 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.631981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:25.632367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:25.632649 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.632907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.633204 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:25.642290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:25.642506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:25.642629 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:25.642739 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:25.643010 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:25.643234 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.643594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:25.643594 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.643594 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:25.643594 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:25.643594 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.643594 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:25.643594 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.643594 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:25.644105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:25.644338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.644338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:25.644338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.644338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:25.644338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:25.644338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.644338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:25.644338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.644338 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.644703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:25.644912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:25.645144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.645144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:25.645144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.645144 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.645373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:25.645708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:25.646025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.646025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:25.646025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.646025 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:25.653793 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:25.654043 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:25.654172 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:25.654280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:25.654632 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:25.654879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:25.655059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.655567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:25.655567 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.655567 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:25.655567 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:25.655567 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.655567 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:25.655567 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.655567 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.656305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.656504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:25.656760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.656760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:25.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:25.657305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:25.657590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:25.657590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:25.657590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:30.377289 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:30.378047 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:30.378171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:30.378778 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:30.405396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:30.405568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.406117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:30.406117 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.406117 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:30.406117 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:30.406117 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.406117 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:30.406117 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.406117 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.406876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.407148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:30.407386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.407386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:30.407591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:30.407926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.408251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:30.408251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.408251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:30.624774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:30.625143 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:30.625280 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:30.626022 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:30.626214 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:30.626335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:30.626447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.626985 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:30.626985 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.626985 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:30.626985 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:30.626985 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.626985 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:30.626985 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.626985 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.628108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.628359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:30.628555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.628555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:30.628934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:30.629222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.629518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:30.629518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.629518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:30.629794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:30.629866 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:30.629946 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:30.630032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:30.630343 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:30.630516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.630912 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:30.630912 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.630912 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:30.630912 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:30.630912 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.630912 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:30.630912 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.630912 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:30.631420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:30.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:30.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:30.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:30.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:30.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.631969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:30.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:30.632394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.632394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:30.632394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.632394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:30.632657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.632913 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:30.633218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.633218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:30.633218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:30.633218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.633218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:30.633218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.633218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:30.642657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:30.642982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:30.643128 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:30.643287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:30.643609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:30.643826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.644296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:30.644296 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.644296 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:30.644296 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:30.644296 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.644296 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:30.644296 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.644296 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:30.645148 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:30.645498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.645498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:30.645498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.645498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:30.645498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:30.645498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.645498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:30.645498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.645498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.645808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:30.646000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:30.646284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.646284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:30.646284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.646284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.646527 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:30.646832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:30.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:30.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:30.654099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:30.654401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:30.654540 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:30.654762 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:30.655056 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:30.655261 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:30.655394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.655860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:30.655860 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.655860 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:30.655860 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:30.655860 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.655860 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:30.655860 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.655860 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.656450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.656692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:30.656882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.656882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:30.657124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:30.657409 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:30.657701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:30.657701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:30.657701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:35.378252 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:35.378731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:35.378731 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:35.379271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:35.403847 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:35.403847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.404053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:35.404053 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.404053 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:35.404053 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:35.404053 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.404053 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:35.404053 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.404053 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.404730 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.404940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:35.405128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.405128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:35.405469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:35.406016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.406458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:35.406458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.406458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:35.625174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:35.625554 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:35.625704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:35.627227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:35.627227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:35.627227 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:35.627227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.627366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:35.627366 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.627366 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:35.627366 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:35.627366 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.627366 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:35.627366 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.627366 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.628036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.628293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:35.628525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.628525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:35.629003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:35.629681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.629964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:35.629964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.629964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:35.629964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:35.630204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:35.630204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:35.630368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:35.630600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:35.630848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.631237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:35.631237 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.631237 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:35.631237 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:35.631237 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.631237 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:35.631237 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.631237 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:35.631745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:35.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.632794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.633006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:35.633211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.633463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:35.633762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.633762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:35.633762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:35.633762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.633762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:35.633762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.633762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:35.642948 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:35.643127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:35.643209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:35.643309 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:35.643982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:35.643982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.644101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:35.644101 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.644101 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:35.644101 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:35.644101 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.644101 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:35.644101 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.644101 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:35.644596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:35.644867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.644867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:35.644867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.644867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:35.644867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:35.644867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.644867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:35.644867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.644867 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:35.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:35.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:35.646416 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.646733 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:35.654377 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:35.654571 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:35.654665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:35.654787 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:35.655510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:35.655510 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:35.655510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.655676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:35.655676 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.655676 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:35.655676 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:35.655676 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.655676 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:35.655676 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.655676 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.656124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.656667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:35.656914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:35.657193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:35.657516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:35.657516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:35.657516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:35.657516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:35.657516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:35.657516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:40.378615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:40.379114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:40.379114 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:40.379715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:40.404769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:40.404769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.404992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:40.404992 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.404992 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:40.404992 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:40.404992 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.404992 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:40.404992 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.404992 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.405691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.406014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:40.406313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.406313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:40.406387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:40.406907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.407339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:40.407339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.407339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:40.625605 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:40.626000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:40.626130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:40.626724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:40.626905 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:40.627017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:40.627127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.627591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:40.627591 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.627591 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:40.627591 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:40.627591 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.627591 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:40.627591 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.627591 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.628397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.628707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:40.628902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.628902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:40.629111 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:40.629436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.629769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:40.629769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.629769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:40.630211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:40.630436 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:40.630515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:40.630629 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:40.630978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:40.631189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.631713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:40.631713 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.631713 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:40.631713 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:40.631713 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.631713 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:40.631713 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.631713 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:40.632220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:40.632555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.632555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:40.632555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.632555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:40.632555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:40.632555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.632555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:40.632555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.632555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.632821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:40.633191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:40.633410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.633410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:40.633410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.633410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:40.633508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.633806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:40.634076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.634076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:40.634076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:40.634076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.634076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:40.634076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.634076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:40.643359 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:40.643615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:40.643723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:40.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:40.644232 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:40.644522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.645160 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:40.645160 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.645160 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:40.645160 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:40.645160 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.645160 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:40.645160 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.645160 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:40.645698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:40.645955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.645955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:40.645955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.645955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:40.645955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:40.645955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.645955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:40.645955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.645955 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.646260 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:40.646468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:40.646699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.646699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:40.646699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.646699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.646969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:40.647379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:40.648343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.648343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:40.648343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.648343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:40.654858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:40.655138 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:40.655219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:40.655365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:40.655857 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:40.656122 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:40.656301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.656915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:40.656915 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.656915 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:40.656915 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:40.656915 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.656915 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:40.656915 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.656915 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.657565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.657961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:40.658301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:40.658720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:40.659214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:40.659214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:40.659214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:40.659214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:40.659214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:40.659214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:45.379089 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:45.379515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:45.379575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:45.380209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:45.405822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:45.405980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.406439 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:45.406439 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.406439 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:45.406439 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:45.406439 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.406439 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:45.406439 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.406439 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.407072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.407349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:45.407564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.407564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:45.407772 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:45.408065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.408398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:45.408398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.408398 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:45.626164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:45.626461 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:45.626557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:45.627197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:45.627384 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:45.627502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:45.627615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.628112 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:45.628112 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.628112 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:45.628112 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:45.628112 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.628112 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:45.628112 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.628112 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.628873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.629154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:45.629393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.629393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:45.629643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:45.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.630327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:45.630327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.630327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:45.630496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:45.630713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:45.630804 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:45.630898 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:45.631146 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:45.631300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.631638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:45.631638 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.631638 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:45.631638 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:45.631638 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.631638 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:45.631638 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.631638 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:45.632153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:45.632435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.632435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:45.632435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.632435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:45.632435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:45.632435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.632435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:45.632435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.632435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.632720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:45.632969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:45.633252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.633252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:45.633252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.633252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:45.633356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.633661 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:45.633987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.633987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:45.633987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:45.633987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.633987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:45.633987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.633987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:45.643675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:45.643874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:45.643964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:45.644064 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:45.644334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:45.644521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.644962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:45.644962 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.644962 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:45.644962 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:45.644962 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.644962 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:45.644962 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.644962 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:45.645475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:45.645899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.645899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:45.645899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.645899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:45.645899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:45.645899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.645899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:45.645899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.645899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.646098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:45.646315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:45.646556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.646556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:45.646556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.646556 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.646839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:45.647156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:45.647520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.647520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:45.647520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.647520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:45.655112 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:45.655323 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:45.655411 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:45.655534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:45.655806 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:45.655952 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:45.656071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.656451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:45.656451 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.656451 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:45.656451 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:45.656451 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.656451 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:45.656451 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.656451 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.657454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:45.657848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:45.658286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:45.658643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:45.658643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:45.658643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:45.658643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:45.658643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:45.658643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:50.379546 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:50.380115 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:50.380115 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:50.380500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:50.407219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:50.407382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.407822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:50.407822 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.407822 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:50.407822 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:50.407822 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.407822 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:50.407822 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.407822 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.408463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.408736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:50.408995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.408995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:50.409228 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:50.409543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.410089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:50.410089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.410089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:50.626552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:50.626986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:50.627117 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:50.627974 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:50.628281 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:50.628483 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:50.628718 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.629377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:50.629377 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.629377 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:50.629377 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:50.629377 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.629377 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:50.629377 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.629377 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.630636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:50.630975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.630975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:50.631296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:50.631768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.632086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:50.632086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.632086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:50.632086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:50.632086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:50.632086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:50.632086 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:50.632472 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:50.632726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.633296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:50.633296 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.633296 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:50.633296 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:50.633296 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.633296 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:50.633296 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.633296 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:50.633884 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:50.634184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.634184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:50.634184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.634184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:50.634184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:50.634184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.634184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:50.634184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.634184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.634496 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:50.634736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:50.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:50.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.634972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:50.635249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.635568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:50.635873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.635873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:50.635873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:50.635873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.635873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:50.635873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.635873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:50.644004 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:50.644258 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:50.644363 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:50.644504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:50.645087 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:50.645370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.645964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:50.645964 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.645964 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:50.645964 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:50.645964 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.645964 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:50.645964 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.645964 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:50.646704 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:50.647122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.647122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:50.647122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.647122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:50.647122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:50.647122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.647122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:50.647122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.647122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.647602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:50.647975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:50.648364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.648364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:50.648364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.648364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.648728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:50.649074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:50.649387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.649387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:50.649387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.649387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:50.655420 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:50.655663 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:50.655792 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:50.655965 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:50.656404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:50.656690 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:50.656874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.657433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:50.657433 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.657433 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:50.657433 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:50.657433 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.657433 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:50.657433 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.657433 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.658223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.658584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:50.659005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.659005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:50.659283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:50.659643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:50.660018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:50.660018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:50.660018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:55.379934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:55.380404 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:55.380579 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:55.381669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:55.383124 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=568,neutron:DELETE=5 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:23:55.410961 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:55.411192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.411763 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:55.411763 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.411763 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:55.411763 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:55.411763 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.411763 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:55.411763 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.411763 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.412459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.412690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:55.412888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.412888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:55.413179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:55.413443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.413814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:55.413814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.413814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:55.627016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:55.627279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:55.627552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:55.628137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:55.628313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:55.628469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:55.628637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.629674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:55.629674 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.629674 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:55.629674 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:55.629674 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.629674 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:55.629674 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.629674 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.630134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.630361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:55.630590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.630590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:55.631199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:55.631477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.631802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:55.631802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.631802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:55.632017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:55.632431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:55.632431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:55.632431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:23:55.632635 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:55.632827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.634671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:55.635235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.635640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.636014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:55.636389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:55.636816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:55.637265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.637265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:55.637265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.637265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:55.637265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:55.637265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.637265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:55.637265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.637265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:55.644338 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:55.644560 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:55.644675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:55.644784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:55.645084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:55.645321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.645742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:55.645742 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.645742 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:55.645742 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:55.645742 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.645742 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:55.645742 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.645742 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:23:55.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:23:55.646501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.646501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:23:55.646501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.646501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:23:55.646501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:23:55.646501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.646501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:23:55.646501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.646501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.646958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:55.647168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:55.647386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.647386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:55.647386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.647386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.647943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:55.648273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:55.648577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.648577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:55.648577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.648577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:23:55.655660 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:23:55.655827 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:23:55.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:23:55.656074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:23:55.656673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:55.656673 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:23:55.656673 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.657019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:23:55.657019 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.657019 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:23:55.657019 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:23:55.657019 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.657019 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:23:55.657019 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.657019 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.657860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.658239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:23:55.658577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.658577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:23:55.658878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:23:55.659171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:23:55.659473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:23:55.659473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:23:55.659473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:00.380396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:00.380809 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:00.380809 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:00.381703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:00.408940 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:00.409076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.409487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:00.409487 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.409487 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:00.409487 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:00.409487 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.409487 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:00.409487 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.409487 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.410097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.410443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:00.410775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.410775 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:00.410841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:00.411293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.411812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:00.411812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.411812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:00.627433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:00.627669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:00.627762 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:00.628262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:00.628415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:00.628527 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:00.628634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.629127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:00.629127 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.629127 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:00.629127 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:00.629127 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.629127 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:00.629127 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.629127 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.629737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.629980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:00.630183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.630183 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:00.630417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:00.630739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.631073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:00.631073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.631073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:00.632241 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:00.632385 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:00.632460 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:00.632541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:00.633095 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:00.633095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.633195 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:00.633195 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.633195 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:00.633195 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:00.633195 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.633195 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:00.633195 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.633195 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:00.633881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:00.634155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.634155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:00.634155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.634155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:00.634155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:00.634155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.634155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:00.634155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.634155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.634750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:00.634965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:00.635174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.635174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:00.635174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.635174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:00.635883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.636151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:00.636459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.636459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:00.636459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:00.636459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.636459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:00.636459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.636459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:00.644582 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:00.644741 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:00.644831 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:00.644937 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:00.645511 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:00.645511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.645612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:00.645612 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.645612 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:00.645612 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:00.645612 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.645612 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:00.645612 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.645612 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:00.646119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:00.646859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:00.647185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:00.647443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:00.647674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.648081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:00.655879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:00.656011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:00.656088 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:00.656193 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:00.656410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:00.656561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:00.656683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:00.656971 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.656971 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:00.656971 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:00.656971 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.656971 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:00.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.657525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.657792 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:00.658000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.658000 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:00.658155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:00.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:00.658818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:00.658818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:00.658818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:05.380791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:05.381275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:05.381275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:05.381737 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:05.408401 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:05.408637 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.409208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:05.409208 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.409208 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:05.409208 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:05.409208 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.409208 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:05.409208 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.409208 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.409963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.410226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:05.410469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.410469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:05.410745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:05.411476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.412093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:05.412093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.412093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:05.627824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:05.628051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:05.628112 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:05.628867 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:05.629044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:05.629197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:05.629292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.629767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:05.629767 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.629767 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:05.629767 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:05.629767 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.629767 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:05.629767 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.629767 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.630530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.630788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:05.631011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.631011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:05.631281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:05.632092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.632607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:05.632607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.632607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:05.632607 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:05.632714 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:05.632714 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:05.632937 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:05.633142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:05.633249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.633666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:05.633666 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.633666 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:05.633666 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:05.633666 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.633666 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:05.633666 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.633666 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:05.634241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:05.634535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.634535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:05.634535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.634535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:05.634535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:05.634535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.634535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:05.634535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.634535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.634928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:05.635321 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:05.635749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.636164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.636725 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:05.644825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:05.645011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:05.645120 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:05.645256 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:05.645599 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:05.645787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.646216 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:05.646216 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.646216 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:05.646216 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:05.646216 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.646216 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:05.646216 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.646216 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:05.646844 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:05.647273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.647273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:05.647273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.647273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:05.647273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:05.647273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.647273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:05.647273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.647273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.647533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:05.647898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:05.648265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.648265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:05.648265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.648265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.648389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:05.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:05.649495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.649495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:05.649495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.649495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:05.656294 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:05.656414 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:05.656553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:05.656856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:05.657048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:05.657211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.657650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:05.657650 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.657650 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:05.657650 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:05.657650 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.657650 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:05.657650 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.657650 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.658331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.658746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:05.659043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.659043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:05.659155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:05.659641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:05.660187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:05.660187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:05.660187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:10.381241 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:10.381825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:10.381825 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:10.382276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:10.408014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:10.408182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.408586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:10.408586 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.408586 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:10.408586 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:10.408586 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.408586 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:10.408586 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.408586 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.409219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.409675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:10.410081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:10.410582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.411213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:10.411213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:10.411213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.411213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:10.411213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.411213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:10.628206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:10.628436 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:10.628532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:10.629265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:10.629473 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:10.629600 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:10.629813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.630220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:10.630220 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.630220 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:10.630220 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:10.630220 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.630220 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:10.630220 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.630220 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.631020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.631310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:10.631521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.631521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:10.632010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:10.632310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.632652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:10.632652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.632652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:10.632755 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:10.632931 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:10.633041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:10.633169 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:10.633467 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:10.633631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.634011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:10.634011 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.634011 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:10.634011 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:10.634011 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.634011 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:10.634011 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.634011 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:10.634679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:10.635218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.635218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:10.635218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.635218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:10.635218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:10.635218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.635218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:10.635218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.635218 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.635574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:10.635964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:10.636293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.636293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:10.636293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.636293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:10.636675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.637103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:10.637577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.637577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:10.637577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:10.637577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.637577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:10.637577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.637577 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:10.645144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:10.645361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:10.645484 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:10.645656 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:10.645966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:10.646145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.646568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:10.646568 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.646568 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:10.646568 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:10.646568 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.646568 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:10.646568 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.646568 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:10.647181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:10.647652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.647652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:10.647652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.647652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:10.647652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:10.647652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.647652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:10.647652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.647652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.647801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:10.648043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:10.648305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.648305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:10.648305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.648305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.648431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:10.648945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:10.649451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.649451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:10.649451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.649451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:10.656463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:10.656725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:10.656826 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:10.656981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:10.657394 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:10.657649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:10.657825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.658266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:10.658266 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.658266 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:10.658266 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:10.658266 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.658266 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:10.658266 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.658266 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.659327 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:10.659684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:10.660129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:10.660743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:10.660743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:10.660743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:10.660743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:10.660743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:10.660743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:15.381809 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:15.382247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:15.382247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:15.382679 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:15.409235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:15.409446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.409943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:15.409943 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.409943 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:15.409943 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:15.409943 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.409943 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:15.409943 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.409943 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.410899 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.411242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:15.411444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.411444 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:15.411750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:15.412319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.412709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:15.412709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.412709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:15.628606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:15.628889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:15.628968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:15.629670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:15.629868 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:15.629991 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:15.630105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.630582 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:15.630582 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.630582 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:15.630582 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:15.630582 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.630582 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:15.630582 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.630582 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.631326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.631526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:15.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.631747 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:15.632168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:15.632751 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:15.633103 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:15.633189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:15.633278 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:15.633494 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:15.633664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.634036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:15.634036 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.634036 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:15.634036 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:15.634036 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.634036 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:15.634036 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.634036 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:15.634579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:15.634833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.634833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:15.634833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.634833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:15.634833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:15.634833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.634833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:15.634833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.634833 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.635169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:15.635379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:15.635591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.635591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:15.635591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.635591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:15.635857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.636341 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:15.636958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.636958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:15.636958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:15.636958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.636958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:15.636958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.636958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:15.645353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:15.645500 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:15.645575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:15.645689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:15.645948 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:15.646104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.646536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:15.646536 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.646536 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:15.646536 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:15.646536 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.646536 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:15.646536 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.646536 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:15.647084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:15.647323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.647323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:15.647323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.647323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:15.647323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:15.647323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.647323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:15.647323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.647323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.647671 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:15.648027 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:15.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:15.648765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.649059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:15.656725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:15.656879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:15.656948 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:15.657065 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:15.657308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:15.657488 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:15.657607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.657994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:15.657994 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.657994 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:15.657994 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:15.657994 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.657994 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:15.657994 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.657994 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.658535 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.658760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:15.658968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.658968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:15.659203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:15.659441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:15.659759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:15.659759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:15.659759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:20.382230 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:20.382703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:20.382703 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:20.383170 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:20.417495 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:20.417767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.418358 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:20.418358 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.418358 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:20.418358 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:20.418358 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.418358 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:20.418358 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.418358 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.419362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.419602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:20.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.419820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:20.420405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:20.420694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.420987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:20.420987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.420987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:20.629134 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:20.629438 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:20.629542 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:20.630352 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:20.630697 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:20.630942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:20.631091 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.631727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:20.631727 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.631727 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:20.631727 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:20.631727 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.631727 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:20.631727 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.631727 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.632506 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.632760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:20.632992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.632992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:20.633208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:20.633480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.634021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:20.634021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.634021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:20.634021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:20.634021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:20.634021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:20.634021 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:20.634187 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:20.634210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.634667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:20.634667 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.634667 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:20.634667 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:20.634667 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.634667 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:20.634667 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.634667 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:20.635213 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:20.635457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.635457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:20.635457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.635457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:20.635457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:20.635457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.635457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:20.635457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.635457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:20.636018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:20.636258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.636258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:20.636258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.636258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:20.636706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.637307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:20.637708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.637708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:20.637708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:20.637708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.637708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:20.637708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.637708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:20.645642 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:20.645846 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:20.645962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:20.646106 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:20.646521 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:20.646780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.647310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:20.647310 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.647310 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:20.647310 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:20.647310 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.647310 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:20.647310 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.647310 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:20.648138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:20.648547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.648547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:20.648547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.648547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:20.648547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:20.648547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.648547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:20.648547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.648547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.649058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:20.649392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:20.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:20.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.650158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:20.650614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:20.651100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.651100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:20.651100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.651100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:20.657027 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:20.657211 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:20.657322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:20.657470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:20.657872 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:20.658104 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:20.658237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.658739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:20.658739 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.658739 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:20.658739 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:20.658739 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.658739 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:20.658739 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.658739 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.659564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.659907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:20.660285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.660285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:20.660563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:20.661105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:20.661534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:20.661534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:20.661534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:25.382670 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:25.383101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:25.383150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:25.383765 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:25.409841 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:25.410078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.410743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:25.410743 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.410743 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:25.410743 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:25.410743 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.410743 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:25.410743 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.410743 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.411451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.411701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:25.411947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.411947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:25.412166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:25.412519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.412941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:25.412941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.412941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:25.629669 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:25.629959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:25.630066 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:25.630643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:25.630802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:25.630905 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:25.631013 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.631461 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:25.631461 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.631461 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:25.631461 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:25.631461 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.631461 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:25.631461 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.631461 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.632452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.632829 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:25.633158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.633158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:25.633375 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:25.633769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.634207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:25.634207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.634207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:25.634207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:25.634207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:25.634207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:25.634207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:25.634207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:25.634423 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.634701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:25.634701 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.634701 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:25.634701 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:25.634701 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.634701 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:25.634701 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.634701 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:25.635229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:25.635657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.635657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:25.635657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.635657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:25.635657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:25.635657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.635657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:25.635657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.635657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.635914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:25.636254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:25.636651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.637023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.637526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:25.645964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:25.646196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:25.646285 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:25.646383 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:25.646636 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:25.646809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.647133 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:25.647133 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.647133 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:25.647133 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:25.647133 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.647133 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:25.647133 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.647133 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:25.647611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:25.648034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.648034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:25.648034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.648034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:25.648034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:25.648034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.648034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:25.648034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.648034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.648281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:25.648598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:25.648988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:25.649386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.650149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:25.657311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:25.657735 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:25.657891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:25.658063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:25.658692 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:25.658843 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:25.658887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.659451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:25.659451 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.659451 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:25.659451 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:25.659451 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.659451 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:25.659451 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.659451 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.660202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.660523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:25.660864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.660864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:25.660990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:25.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:25.661858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:25.661858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:25.661858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:30.383173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:30.383610 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:30.383704 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:30.384349 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:30.412153 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:30.412275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.412804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:30.412804 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.412804 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:30.412804 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:30.412804 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.412804 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:30.412804 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.412804 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.413478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.413758 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:30.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.413974 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:30.414232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:30.414531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.414851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:30.414851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.414851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:30.629999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:30.630397 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:30.630489 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:30.631096 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:30.631274 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:30.631370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:30.631473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.631914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:30.631914 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.631914 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:30.631914 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:30.631914 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.631914 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:30.631914 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.631914 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.632566 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.632865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:30.633185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.633185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:30.633381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:30.633862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.634435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:30.634435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.634435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:30.634435 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:30.634435 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:30.634435 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:30.634435 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:30.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:30.634688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.635030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:30.635030 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.635030 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:30.635030 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:30.635030 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.635030 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:30.635030 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.635030 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:30.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:30.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:30.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:30.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:30.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:30.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.636274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:30.636644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:30.637062 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.637487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.638048 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:30.646425 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:30.646710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:30.646805 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:30.646910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:30.647160 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:30.647339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.647729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:30.647729 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.647729 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:30.647729 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:30.647729 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.647729 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:30.647729 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.647729 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:30.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:30.648786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.648786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:30.648786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.648786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:30.648786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:30.648786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.648786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:30.648786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.648786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.649053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:30.649402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:30.649813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:30.650197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.650708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:30.657773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:30.658032 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:30.658130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:30.658287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:30.658509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:30.658694 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:30.658803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.659135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:30.659135 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.659135 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:30.659135 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:30.659135 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.659135 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:30.659135 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.659135 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.659722 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:30.660437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:30.660876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:30.661380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:30.661380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:30.661380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:30.661380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:30.661380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:30.661380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:35.383798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:35.384305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:35.384305 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:35.384803 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:35.408944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:35.409173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.409692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:35.409692 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.409692 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:35.409692 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:35.409692 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.409692 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:35.409692 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.409692 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.410431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.410859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:35.411222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:35.411699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.412222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:35.412222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:35.412222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.412222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:35.412222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.412222 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:35.630532 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:35.630906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:35.630992 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:35.631525 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:35.631678 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:35.631784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:35.631876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.632307 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:35.632307 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.632307 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:35.632307 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:35.632307 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.632307 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:35.632307 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.632307 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.633034 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.633408 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:35.633754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:35.634185 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.634689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:35.634689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:35.634689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.634689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:35.634689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.634689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:35.634689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:35.634689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:35.634689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:35.634689 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:35.634979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:35.635028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.635355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:35.635355 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.635355 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:35.635355 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:35.635355 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.635355 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:35.635355 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.635355 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:35.635895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:35.636328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.636328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:35.636328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.636328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:35.636328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:35.636328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.636328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:35.636328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.636328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.636592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:35.636929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:35.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.637663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.638142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:35.647000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:35.647265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:35.647359 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:35.647505 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:35.647772 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:35.647932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.648254 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:35.648254 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.648254 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:35.648254 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:35.648254 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.648254 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:35.648254 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.648254 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:35.648780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:35.649168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.649168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:35.649168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.649168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:35.649168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:35.649168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.649168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:35.649168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.649168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.649389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:35.649723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:35.650069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.650069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:35.650069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.650069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.650176 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:35.650616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:35.651083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.651083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:35.651083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.651083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:35.658290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:35.658555 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:35.658659 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:35.658801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:35.659037 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:35.659216 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:35.659343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.659741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:35.659741 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.659741 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:35.659741 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:35.659741 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.659741 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:35.659741 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.659741 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.660294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.660653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:35.660978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:35.661362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:35.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:35.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:35.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:35.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:35.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:35.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:40.384348 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:40.384890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:40.384890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:40.386014 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:40.415241 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:40.415699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.416415 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:40.416415 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.416415 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:40.416415 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:40.416415 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.416415 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:40.416415 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.416415 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.418146 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.418432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:40.418687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:40.418931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.419214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:40.419214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:40.419214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.419214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:40.419214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.419214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:40.631076 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:40.631403 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:40.631506 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:40.632205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:40.632410 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:40.632552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:40.632723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.633181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:40.633181 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.633181 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:40.633181 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:40.633181 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.633181 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:40.633181 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.633181 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.633934 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.634178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:40.634641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.634641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:40.635031 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:40.635442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.635812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:40.635812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.635812 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:40.635812 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:40.635812 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:40.635812 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:40.635812 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:40.635812 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:40.635971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.636391 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:40.636391 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.636391 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:40.636391 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:40.636391 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.636391 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:40.636391 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.636391 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:40.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:40.637101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.637101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:40.637101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.637101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:40.637101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:40.637101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.637101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:40.637101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.637101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.637438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:40.637710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:40.637939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.637939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:40.637939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.637939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:40.638359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.638753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:40.639018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.639018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:40.639018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:40.639018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.639018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:40.639018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.639018 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:40.647426 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:40.647754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:40.647852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:40.647950 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:40.648282 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:40.648470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.648951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:40.648951 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.648951 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:40.648951 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:40.648951 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.648951 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:40.648951 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.648951 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:40.649590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:40.649927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.649927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:40.649927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.649927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:40.649927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:40.649927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.649927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:40.649927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.649927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.650248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:40.650679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:40.650964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.650964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:40.650964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.650964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.651246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:40.651748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:40.652023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.652023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:40.652023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.652023 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:40.658726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:40.658979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:40.659073 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:40.659264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:40.659611 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:40.659817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:40.659959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.660410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:40.660410 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.660410 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:40.660410 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:40.660410 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.660410 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:40.660410 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.660410 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.661065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.661426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:40.662009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:40.662800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:40.663129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:40.663129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:40.663129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:40.663129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:40.663129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:40.663129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:45.384823 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:45.385245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:45.385245 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:45.385917 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:45.409212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:45.409405 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.409937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:45.409937 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.409937 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:45.409937 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:45.409937 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.409937 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:45.409937 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.409937 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.410583 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.410980 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:45.411274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:45.411568 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.411945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:45.411945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:45.411945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.411945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:45.411945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.411945 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:45.631425 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:45.631824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:45.631939 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:45.632773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:45.633013 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:45.633168 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:45.633312 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.633848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:45.633848 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.633848 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:45.633848 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:45.633848 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.633848 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:45.633848 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.633848 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.634541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.634861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:45.635074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.635074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:45.635275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:45.635561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.635891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:45.635891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.635891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:45.635891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:45.635891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:45.635891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:45.635891 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:45.636279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:45.636430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.636814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:45.636814 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.636814 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:45.636814 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:45.636814 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.636814 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:45.636814 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.636814 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:45.637340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:45.637639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.637639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:45.637639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.637639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:45.637639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:45.637639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.637639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:45.637639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.637639 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.637983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:45.638208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:45.638437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.638437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:45.638437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.638437 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:45.638700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.638981 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:45.639275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.639275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:45.639275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:45.639275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.639275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:45.639275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.639275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:45.647698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:45.648009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:45.648113 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:45.648225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:45.648589 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:45.648822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.649210 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:45.649210 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.649210 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:45.649210 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:45.649210 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.649210 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:45.649210 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.649210 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:45.649754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:45.650039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.650039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:45.650039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.650039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:45.650039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:45.650039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.650039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:45.650039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.650039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.650351 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:45.650557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:45.650804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.650804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:45.650804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.650804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.651073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:45.651456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:45.651782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.651782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:45.651782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.651782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:45.658996 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:45.659271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:45.659357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:45.659494 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:45.659782 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:45.660017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:45.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.660597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:45.660597 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.660597 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:45.660597 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:45.660597 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.660597 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:45.660597 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.660597 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.661200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.661401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:45.661615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.661615 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:45.661851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:45.662101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:45.662410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:45.662410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:45.662410 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:50.385277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:50.385728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:50.385728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:50.386319 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:50.412288 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:50.412549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.413244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:50.413244 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.413244 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:50.413244 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:50.413244 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.413244 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:50.413244 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.413244 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.414009 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.414252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:50.414591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.414591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:50.414969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:50.415247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.415638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:50.415638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.415638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:50.631902 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:50.632263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:50.632357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:50.633058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:50.633322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:50.633475 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:50.633644 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.634236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:50.634236 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.634236 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:50.634236 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:50.634236 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.634236 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:50.634236 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.634236 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.634963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.635263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:50.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.635526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:50.635678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:50.635986 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.636273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:50.636273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.636273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:50.636273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:50.636273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:50.636273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:50.636273 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:50.636509 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:50.636663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.637030 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:50.637030 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.637030 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:50.637030 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:50.637030 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.637030 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:50.637030 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.637030 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:50.637532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:50.638010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.638010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:50.638010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.638010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:50.638010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:50.638010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.638010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:50.638010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.638010 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.638275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:50.638643 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:50.639026 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.639386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.639889 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:50.647982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:50.648155 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:50.648233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:50.648329 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:50.648614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:50.648811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.649156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:50.649156 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.649156 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:50.649156 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:50.649156 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.649156 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:50.649156 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.649156 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:50.649678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:50.650124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.650124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:50.650124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.650124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:50.650124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:50.650124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.650124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:50.650124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.650124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.650363 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:50.650699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:50.651068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:50.651522 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.652052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:50.659297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:50.659492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:50.659576 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:50.659714 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:50.659998 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:50.660163 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:50.660282 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.660638 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:50.660638 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.660638 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:50.660638 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:50.660638 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.660638 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:50.660638 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.660638 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.661220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.661490 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:50.661727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.661727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:50.661946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:50.662209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:50.662548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:50.662548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:50.662548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:55.385614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:55.386166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:55.386166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:55.386874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:55.391180 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=241 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:24:55.414919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:55.415200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.415811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:55.415811 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.415811 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:55.415811 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:55.415811 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.415811 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:55.415811 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.415811 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.416778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.417099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:55.417325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.417325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:55.417580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:55.417898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.418215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:55.418215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.418215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:55.632393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:55.632815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:55.632930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:55.633716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:55.633942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:55.634101 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:55.634221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.634741 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:55.634741 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.634741 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:55.634741 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:55.634741 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.634741 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:55.634741 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.634741 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.635520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.635805 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:55.636078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.636078 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:55.636311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:55.636717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.637083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:55.637083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.637083 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:55.637083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:55.637083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:55.637083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:55.637204 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:24:55.637428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:55.637576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.638059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:55.638059 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.638059 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:55.638059 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:55.638059 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.638059 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:55.638059 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.638059 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:55.638590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:55.638861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.638861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:55.638861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.638861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:55.638861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:55.638861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.638861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:55.638861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.638861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.639173 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:55.639394 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:55.639645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.639645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:55.639645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.639645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:55.639843 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.640181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:55.640483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.640483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:55.640483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:55.640483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.640483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:55.640483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.640483 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:55.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:55.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:55.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:55.648960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:55.649311 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:55.649495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.649964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:55.649964 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.649964 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:55.649964 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:55.649964 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.649964 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:55.649964 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.649964 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:24:55.650931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:24:55.651227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.651227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:24:55.651227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.651227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:24:55.651227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:24:55.651227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.651227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:24:55.651227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.651227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.651922 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:55.652149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:55.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:55.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.653076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:55.653384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:55.653749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.653749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:55.653749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.653749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:24:55.659769 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:24:55.660110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:24:55.660240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:24:55.660421 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:24:55.660805 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:55.661008 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:24:55.661140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.661592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:24:55.661592 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.661592 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:24:55.661592 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:24:55.661592 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.661592 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:24:55.661592 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.661592 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.662633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.663022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:24:55.663385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.663385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:24:55.663723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:24:55.664052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:24:55.664334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:24:55.664334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:24:55.664334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:00.386029 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:00.386455 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:00.386455 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:00.387198 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:00.419727 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:00.419963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.420537 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:00.420537 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.420537 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:00.420537 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:00.420537 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.420537 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:00.420537 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.420537 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.421296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.421561 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:00.421818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.421818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:00.422005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:00.422303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.422712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:00.422712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.422712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:00.632899 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:00.633126 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:00.633221 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:00.633955 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:00.634195 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:00.634317 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:00.634432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.634901 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:00.634901 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.634901 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:00.634901 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:00.634901 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.634901 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:00.634901 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.634901 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.635586 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.635978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:00.636386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:00.636850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.637381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:00.637381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:00.637381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.637381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:00.637381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.637381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:00.637381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:00.637381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:00.637381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:00.637381 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:00.637693 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:00.637693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.638070 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:00.638070 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.638070 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:00.638070 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:00.638070 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.638070 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:00.638070 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.638070 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:00.638630 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:00.639061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.639061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:00.639061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.639061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:00.639061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:00.639061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.639061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:00.639061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.639061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.639328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:00.639677 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:00.640094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.640486 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.641006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:00.648667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:00.648822 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:00.648958 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:00.649001 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:00.649268 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:00.649434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.649822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:00.649822 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.649822 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:00.649822 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:00.649822 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.649822 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:00.649822 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.649822 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:00.650377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:00.650852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.650852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:00.650852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.650852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:00.650852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:00.650852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.650852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:00.650852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.650852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.651211 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:00.651482 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:00.651765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.651765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:00.651765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.651765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.651859 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:00.652193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:00.652518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.652518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:00.652518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.652518 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:00.660067 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:00.660267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:00.660353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:00.660468 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:00.660750 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:00.660919 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:00.661038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.661466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:00.661466 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.661466 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:00.661466 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:00.661466 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.661466 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:00.661466 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.661466 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.662058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.662275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:00.662469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.662469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:00.662766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:00.663061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:00.663343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:00.663343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:00.663343 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:05.386462 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:05.386930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:05.386930 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:05.387501 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:05.413275 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:05.413448 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.413877 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:05.413877 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.413877 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:05.413877 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:05.413877 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.413877 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:05.413877 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.413877 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.414549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.414796 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:05.414992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.414992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:05.415459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:05.415774 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.416094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:05.416094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.416094 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:05.633306 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:05.633594 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:05.633738 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:05.634428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:05.634615 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:05.634773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:05.634915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.635435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:05.635435 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.635435 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:05.635435 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:05.635435 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.635435 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:05.635435 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.635435 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.636240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.636633 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:05.637024 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:05.637476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.638044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:05.638044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:05.638044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.638044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:05.638044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.638044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:05.638044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:05.638044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:05.638044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:05.638044 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:05.638588 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:05.638776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.639187 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:05.639187 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.639187 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:05.639187 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:05.639187 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.639187 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:05.639187 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.639187 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:05.639770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:05.640197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.640197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:05.640197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.640197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:05.640197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:05.640197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.640197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:05.640197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.640197 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.640468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:05.640853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:05.641219 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.641652 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.642208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:05.648890 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:05.649081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:05.649178 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:05.649290 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:05.649711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:05.649988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.650495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:05.650495 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.650495 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:05.650495 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:05.650495 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.650495 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:05.650495 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.650495 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:05.651071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:05.651470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.651470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:05.651470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.651470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:05.651470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:05.651470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.651470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:05.651470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.651470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.651724 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:05.652011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:05.652371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:05.652825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.653378 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:05.660353 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:05.660541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:05.660652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:05.660780 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:05.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:05.661421 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:05.661585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.662074 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:05.662074 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.662074 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:05.662074 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:05.662074 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.662074 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:05.662074 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.662074 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.662720 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.663103 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:05.663495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:05.663926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:05.664465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:05.664465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:05.664465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:05.664465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:05.664465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:05.664465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:10.386896 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:10.387480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:10.387480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:10.387972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:10.414376 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:10.414700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.415277 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:10.415277 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.415277 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:10.415277 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:10.415277 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.415277 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:10.415277 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.415277 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.415983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.416267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:10.416558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.416558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:10.416821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:10.417465 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.418220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:10.418220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.418220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:10.633723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:10.633935 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:10.634054 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:10.634856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:10.635084 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:10.635243 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:10.635386 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.635863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:10.635863 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.635863 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:10.635863 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:10.635863 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.635863 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:10.635863 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.635863 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.636778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.637064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:10.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.637283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:10.637650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:10.637928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.638291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:10.638291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.638291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:10.638291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:10.638291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:10.638291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:10.638291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:10.638754 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:10.638754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.639203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:10.639203 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.639203 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:10.639203 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:10.639203 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.639203 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:10.639203 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.639203 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:10.639866 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:10.640640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.640640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:10.640640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.640640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:10.640640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:10.640640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.640640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:10.640640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.640640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.640872 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:10.641147 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:10.641438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.641789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.642199 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:10.649210 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:10.649423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:10.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:10.649629 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:10.650132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:10.650382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.650900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:10.650900 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.650900 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:10.650900 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:10.650900 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.650900 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:10.650900 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.650900 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:10.651808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:10.652073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.652073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:10.652073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.652073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:10.652073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:10.652073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.652073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:10.652073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.652073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:10.652707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:10.652948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.652948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:10.652948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.652948 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.653372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:10.653807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:10.654114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.654114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:10.654114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.654114 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:10.660613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:10.660821 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:10.660908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:10.661049 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:10.661539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:10.661836 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:10.662015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.662595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:10.662595 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.662595 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:10.662595 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:10.662595 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.662595 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:10.662595 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.662595 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.663305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.663636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:10.663842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.663842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:10.664075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:10.664336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:10.664664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:10.664664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:10.664664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:15.387294 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:15.387878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:15.387878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:15.388356 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:15.414262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:15.414507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.415113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:15.415113 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.415113 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:15.415113 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:15.415113 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.415113 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:15.415113 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.415113 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.416214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.416731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:15.417076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.417076 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:15.417679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:15.418675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.419382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:15.419382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.419382 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:15.634127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:15.634399 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:15.634561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:15.635217 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:15.635416 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:15.635584 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:15.635760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.636305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:15.636305 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.636305 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:15.636305 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:15.636305 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.636305 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:15.636305 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.636305 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.637097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.637478 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:15.637839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.637839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:15.638227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:15.638675 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.639119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:15.639119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.639119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:15.639119 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:15.639119 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:15.639119 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:15.639119 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:15.639236 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:15.639505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.639806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:15.639806 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.639806 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:15.639806 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:15.639806 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.639806 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:15.639806 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.639806 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:15.640487 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:15.641477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.642093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.642443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:15.642714 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.643005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:15.643313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.643313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:15.643313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:15.643313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.643313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:15.643313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.643313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:15.649541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:15.649797 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:15.649923 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:15.650078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:15.650413 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:15.650683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.651186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:15.651186 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.651186 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:15.651186 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:15.651186 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.651186 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:15.651186 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.651186 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:15.651764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:15.652068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.652068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:15.652068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.652068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:15.652068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:15.652068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.652068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:15.652068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.652068 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.652377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:15.652588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:15.652817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.652817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:15.652817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.652817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.653267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:15.653611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:15.653937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.653937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:15.653937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.653937 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:15.661016 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:15.661263 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:15.661364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:15.661531 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:15.661878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:15.662149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:15.662323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.662886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:15.662886 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.662886 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:15.662886 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:15.662886 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.662886 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:15.662886 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.662886 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.663614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.663881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:15.664118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.664118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:15.664433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:15.665129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:15.665688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:15.665688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:15.665688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:20.387808 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:20.388320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:20.388320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:20.389099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:20.423228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:20.423484 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.424208 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:20.424208 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.424208 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:20.424208 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:20.424208 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.424208 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:20.424208 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.424208 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.425477 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.425879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:20.426084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.426084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:20.426781 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:20.427139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.427542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:20.427542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.427542 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:20.634595 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:20.635023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:20.635127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:20.635936 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:20.636228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:20.636458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:20.636699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.637347 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:20.637347 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.637347 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:20.637347 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:20.637347 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.637347 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:20.637347 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.637347 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.638162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.638426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:20.638663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.638663 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:20.638854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:20.639167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.639476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:20.639476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.639476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:20.639476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:20.639569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:20.639741 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:20.639852 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:20.640228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:20.640462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.641065 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:20.641065 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.641065 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:20.641065 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:20.641065 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.641065 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:20.641065 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.641065 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:20.641689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:20.642055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.642055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:20.642055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.642055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:20.642055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:20.642055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.642055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:20.642055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.642055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.642585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:20.643151 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:20.643458 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.643827 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.644266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:20.649925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:20.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:20.650577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:20.650577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:20.651129 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:20.651471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.652245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:20.652245 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.652245 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:20.652245 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:20.652245 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.652245 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:20.652245 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.652245 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:20.652947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:20.653239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.653239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:20.653239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.653239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:20.653239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:20.653239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.653239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:20.653239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.653239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.653593 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:20.653860 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:20.654067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.654067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:20.654067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.654067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.654385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:20.654717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:20.655118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.655118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:20.655118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.655118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:20.661298 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:20.661491 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:20.661597 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:20.661775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:20.662218 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:20.662502 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:20.662727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.663299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:20.663299 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.663299 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:20.663299 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:20.663299 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.663299 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:20.663299 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.663299 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.663976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.664251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:20.664493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.664493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:20.664739 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:20.665085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:20.665438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:20.665438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:20.665438 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:25.388199 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:25.388718 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:25.388718 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:25.389335 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:25.418470 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:25.418692 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.419334 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:25.419334 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.419334 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:25.419334 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:25.419334 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.419334 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:25.419334 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.419334 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.420080 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.420360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:25.420552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.420552 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:25.420964 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:25.421525 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.421870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:25.421870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.421870 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:25.635083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:25.635437 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:25.635563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:25.636314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:25.636518 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:25.636667 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:25.636806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.637279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:25.637279 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.637279 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:25.637279 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:25.637279 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.637279 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:25.637279 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.637279 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.638060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.638365 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:25.638607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.638607 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:25.638822 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:25.639155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.639479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:25.639479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.639479 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:25.639583 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:25.639776 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:25.639856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:25.639951 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:25.640243 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:25.640390 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.640830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:25.640830 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.640830 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:25.640830 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:25.640830 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.640830 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:25.640830 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.640830 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:25.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:25.641594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.641594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:25.641594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.641594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:25.641594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:25.641594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.641594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:25.641594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.641594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.641969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:25.642189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:25.642456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.642456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:25.642456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.642456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:25.642660 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.642951 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:25.643250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.643250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:25.643250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:25.643250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.643250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:25.643250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.643250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:25.650226 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:25.650379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:25.650480 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:25.650577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:25.650927 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:25.651132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.651562 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:25.651562 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.651562 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:25.651562 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:25.651562 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.651562 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:25.651562 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.651562 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:25.652119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:25.652564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.652564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:25.652564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.652564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:25.652564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:25.652564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.652564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:25.652564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.652564 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.652839 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:25.653126 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:25.653495 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:25.653904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.654389 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:25.661549 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:25.661724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:25.661824 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:25.661984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:25.662346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:25.662557 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:25.662740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.663209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:25.663209 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.663209 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:25.663209 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:25.663209 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.663209 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:25.663209 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.663209 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.664086 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:25.664323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.664323 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:25.664429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:25.664757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:25.665100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:25.665100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:25.665100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:30.388686 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:30.389237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:30.389237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:30.389918 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:30.416121 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:30.416284 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.416706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:30.416706 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.416706 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:30.416706 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:30.416706 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.416706 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:30.416706 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.416706 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.417513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.417883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:30.418214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.418214 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:30.418591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:30.418902 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.419192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:30.419192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.419192 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:30.635449 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:30.635695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:30.635779 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:30.636895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:30.636895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:30.636895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:30.636895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.637115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:30.637115 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.637115 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:30.637115 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:30.637115 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.637115 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:30.637115 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.637115 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.637699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.637940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:30.638137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.638137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:30.638380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:30.638655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.638999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:30.638999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.638999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:30.639840 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:30.640005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:30.640081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:30.640174 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:30.640428 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:30.640616 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.641019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:30.641019 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.641019 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:30.641019 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:30.641019 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.641019 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:30.641019 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.641019 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:30.641471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:30.641735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.641735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:30.641735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.641735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:30.641735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:30.641735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.641735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:30.641735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.641735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.642038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:30.642687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:30.643060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.643291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.643570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:30.650541 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:30.650775 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:30.650854 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:30.650982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:30.651276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:30.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.651880 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:30.651880 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.651880 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:30.651880 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:30.651880 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.651880 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:30.651880 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.651880 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:30.652413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:30.652678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.652678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:30.652678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.652678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:30.652678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:30.652678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.652678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:30.652678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.652678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.653001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:30.653370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:30.653809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:30.654047 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.654326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:30.661887 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:30.662127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:30.662206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:30.662341 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:30.662634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:30.662878 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:30.663155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.663634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:30.663634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.663634 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:30.663634 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:30.663634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.663634 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:30.663634 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.663634 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.664249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.664494 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:30.664824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.664824 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:30.665081 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:30.665707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:30.666291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:30.666291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:30.666291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:35.389025 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:35.389586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:35.389586 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:35.390257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:35.413476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:35.413689 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.414166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:35.414166 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.414166 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:35.414166 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:35.414166 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.414166 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:35.414166 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.414166 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.414927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.415315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:35.415707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.415707 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:35.415995 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:35.416305 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.416602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:35.416602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.416602 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:35.635956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:35.636368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:35.636484 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:35.637206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:35.637412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:35.637528 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:35.637679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.638161 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:35.638161 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.638161 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:35.638161 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:35.638161 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.638161 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:35.638161 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.638161 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.639366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:35.639749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.639749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:35.640085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:35.640456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.640994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:35.640994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.640994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:35.640994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:35.640994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:35.640994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:35.640994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:35.641135 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:35.641247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.641632 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:35.641632 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.641632 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:35.641632 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:35.641632 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.641632 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:35.641632 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.641632 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:35.642293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:35.642713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.642713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:35.642713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.642713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:35.642713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:35.642713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.642713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:35.642713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.642713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.643124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:35.643453 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:35.643842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.643842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:35.643842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.643842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:35.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.644475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:35.644832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.644832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:35.644832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:35.644832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.644832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:35.644832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.644832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:35.650832 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:35.651097 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:35.651210 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:35.651363 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:35.651695 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:35.651891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.652309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:35.652309 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.652309 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:35.652309 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:35.652309 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.652309 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:35.652309 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.652309 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:35.653002 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:35.653387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.653387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:35.653387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.653387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:35.653387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:35.653387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.653387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:35.653387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.653387 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.653935 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:35.654248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:35.654588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.654588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:35.654588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.654588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.655006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:35.655367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:35.655654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.655654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:35.655654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.655654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:35.662219 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:35.662441 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:35.662553 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:35.662740 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:35.663057 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:35.663240 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:35.663361 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:35.663800 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.663800 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:35.663800 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:35.663800 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.663800 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:35.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.663800 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.664549 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.664890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:35.665255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.665255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:35.665636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:35.665943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:35.666229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:35.666229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:35.666229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:40.389474 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:40.389978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:40.389978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:40.390654 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:40.418698 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:40.418925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.419579 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:40.419579 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.419579 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:40.419579 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:40.419579 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.419579 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:40.419579 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.419579 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.420336 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.420696 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:40.420944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.420944 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:40.421179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:40.421571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.421932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:40.421932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.421932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:40.636388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:40.636660 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:40.636801 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:40.637521 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:40.637735 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:40.637874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:40.637989 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.638469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:40.638469 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.638469 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:40.638469 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:40.638469 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.638469 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:40.638469 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.638469 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.639300 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.639729 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:40.640109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.640109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:40.640371 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:40.640808 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.641279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:40.641279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.641279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:40.641279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:40.641279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:40.641279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:40.641279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:40.641279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:40.641450 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.641815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:40.641815 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.641815 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:40.641815 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:40.641815 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.641815 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:40.641815 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.641815 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:40.642442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:40.642925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.642925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:40.642925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.642925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:40.642925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:40.642925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.642925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:40.642925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.642925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.643333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:40.643712 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:40.644124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.644124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:40.644124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.644124 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:40.644395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.644776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:40.645071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.645071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:40.645071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:40.645071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.645071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:40.645071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.645071 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:40.651080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:40.651303 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:40.651451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:40.651616 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:40.652197 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:40.652473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.653038 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:40.653038 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.653038 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:40.653038 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:40.653038 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.653038 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:40.653038 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.653038 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:40.653678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:40.654117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.654117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:40.654117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.654117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:40.654117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:40.654117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.654117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:40.654117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.654117 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.654563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:40.654910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:40.655313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.655313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:40.655313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.655313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.655658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:40.655990 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:40.656309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.656309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:40.656309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.656309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:40.662435 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:40.662676 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:40.662815 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:40.662966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:40.663418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:40.663684 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:40.663863 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.664345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:40.664345 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.664345 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:40.664345 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:40.664345 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.664345 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:40.664345 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.664345 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.665056 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.665421 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:40.665826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.665826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:40.666058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:40.666406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:40.666778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:40.666778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:40.666778 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:45.389982 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:45.390959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:45.390959 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:45.391504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:45.419908 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:45.420084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.420569 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:45.420569 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.420569 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:45.420569 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:45.420569 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.420569 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:45.420569 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.420569 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.421422 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.421757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:45.421982 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:45.422239 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.422588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:45.422588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:45.422588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.422588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:45.422588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.422588 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:45.636906 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:45.637602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:45.637602 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:45.638388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:45.638388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:45.638388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:45.638388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.638923 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:45.638923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.638923 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:45.638923 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:45.638923 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.638923 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:45.638923 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.638923 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.639890 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.640278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:45.640683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.640683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:45.641093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:45.641532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.642099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:45.642099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.642099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:45.642099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:45.642099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:45.642099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:45.642099 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:45.642561 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:45.642821 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.643433 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:45.643433 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.643433 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:45.643433 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:45.643433 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.643433 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:45.643433 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.643433 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:45.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:45.644968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.645207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.645536 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:45.645878 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.646154 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:45.646534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.646534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:45.646534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:45.646534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.646534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:45.646534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.646534 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:45.651388 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:45.651640 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:45.651766 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:45.651903 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:45.652283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:45.652463 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.652917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:45.652917 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.652917 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:45.652917 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:45.652917 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.652917 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:45.652917 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.652917 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:45.653684 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:45.653978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.653978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:45.653978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.653978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:45.653978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:45.653978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.653978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:45.653978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.653978 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.654546 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:45.654804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:45.655140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.655140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:45.655140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.655140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.655431 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:45.655848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:45.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:45.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:45.662792 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:45.662984 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:45.663064 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:45.663228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:45.663527 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:45.664064 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:45.664194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.664669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:45.664669 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.664669 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:45.664669 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:45.664669 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.664669 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:45.664669 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.664669 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.665298 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.665545 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:45.665782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.665782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:45.666006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:45.666296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:45.666604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:45.666604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:45.666604 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:50.390773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:50.391215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:50.391215 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:50.391712 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:50.415418 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:50.415565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.415967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:50.415967 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.415967 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:50.415967 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:50.415967 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.415967 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:50.415967 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.415967 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.416544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.416817 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:50.417043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.417043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:50.417278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:50.417587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:50.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.417926 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:50.637407 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:50.637638 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:50.637743 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:50.638314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:50.638469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:50.638575 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:50.638691 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.639127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:50.639127 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.639127 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:50.639127 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:50.639127 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.639127 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:50.639127 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.639127 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.639737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.639941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:50.640155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.640155 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:50.640488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:50.640765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.641097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:50.641097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.641097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:50.641614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:50.641979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:50.642075 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:50.642166 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:50.642463 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:50.642601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.642993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:50.642993 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.642993 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:50.642993 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:50.642993 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.642993 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:50.642993 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.642993 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:50.643608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:50.643869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.643869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:50.643869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.643869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:50.643869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:50.643869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.643869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:50.643869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.643869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.644166 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:50.644399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:50.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:50.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.644666 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:50.644854 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.645123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:50.645413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.645413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:50.645413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:50.645413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.645413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:50.645413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.645413 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:50.651811 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:50.652109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:50.652206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:50.652334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:50.652711 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:50.653039 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.653468 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:50.653468 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.653468 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:50.653468 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:50.653468 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.653468 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:50.653468 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.653468 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:50.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:50.654471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.654471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:50.654471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.654471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:50.654471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:50.654471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.654471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:50.654471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.654471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.654757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:50.655088 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:50.655489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:50.655930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.656452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:50.663130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:50.663332 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:50.663423 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:50.663534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:50.663835 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:50.664031 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:50.664186 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.664635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:50.664635 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.664635 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:50.664635 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:50.664635 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.664635 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:50.664635 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.664635 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.665178 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.665541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:50.665848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.665848 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:50.665938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:50.666224 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:50.666576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:50.666576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:50.666576 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:55.391229 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:55.391860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:55.391860 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:55.392508 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:55.397267 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=240 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:25:55.431614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:55.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:55.432473 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.432473 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:55.432473 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:55.432473 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.432473 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:55.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.432473 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.433557 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.433823 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:55.434054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.434054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:55.434767 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:55.435064 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.435355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:55.435355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.435355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:55.637834 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:55.638239 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:55.638361 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:55.639005 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:55.639130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:55.639237 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:55.639350 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.639788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:55.639788 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.639788 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:55.639788 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:55.639788 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.639788 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:55.639788 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.639788 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.640388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.640635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:55.640840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.640840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:55.641096 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:55.641397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.641765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:55.641765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.641765 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:55.641934 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:55.642136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:55.642230 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:55.642321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:25:55.642548 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:55.642776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.643179 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:55.643179 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.643179 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:55.643179 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:55.643179 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.643179 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:55.643179 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.643179 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:55.643892 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:55.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:55.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:55.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:55.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:55.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.644149 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.644533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:55.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:55.644966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.644966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:55.644966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.644966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:55.645268 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.645532 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:55.645840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.645840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:55.645840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:55.645840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.645840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:55.645840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.645840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:55.652207 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:55.652412 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:55.652534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:55.652761 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:55.653078 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:55.653329 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.653903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:55.653903 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.653903 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:55.653903 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:55.653903 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.653903 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:55.653903 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.653903 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:25:55.654655 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:25:55.655180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.655180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:25:55.655180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.655180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:25:55.655180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:25:55.655180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.655180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:25:55.655180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.655180 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.655474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:55.655845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:55.656253 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:55.656753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.657316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:25:55.663444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:25:55.664257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:25:55.664257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:25:55.664257 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:25:55.664524 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:55.664788 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:25:55.664988 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.665690 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:25:55.665690 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.665690 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:25:55.665690 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:25:55.665690 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.665690 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:25:55.665690 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.665690 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.666520 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.666806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:25:55.667045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.667045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:25:55.667938 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:25:55.668221 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:25:55.668555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:25:55.668555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:25:55.668555 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:00.391699 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:00.392264 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:00.392321 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:00.393246 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:00.419783 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:00.419997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.420481 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:00.420481 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.420481 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:00.420481 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:00.420481 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.420481 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:00.420481 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.420481 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.421430 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:00.421921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.421921 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:00.422581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:00.422857 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.423171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:00.423171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.423171 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:00.638390 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:00.639442 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:00.639442 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:00.639719 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:00.639883 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cb0982dc-f84e-421b-ae33-411ef007397f, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:00.639999 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DelStaticRoutesCommand(_result=None, lrouter=neutron-dbce49bb-17ac-40db-91aa-b0d13e58bb7b, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:00.640101 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.640698 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:00.640698 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.640698 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:00.640698 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:00.640698 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.640698 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:00.640698 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.640698 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.641417 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.641861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:00.642123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.642123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:00.642435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:00.642883 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.643196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:00.643196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.643196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:00.643196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:00.643196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:00.643308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:00.643342 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:00.643634 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:00.643769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.644118 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:00.644118 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.644118 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:00.644118 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:00.644118 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.644118 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:00.644118 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.644118 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:00.644735 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:00.645020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.645020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:00.645020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.645020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:00.645020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:00.645020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.645020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:00.645020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.645020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.645401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:00.645636 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:00.645876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.645876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:00.645876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.645876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:00.646263 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.646611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:00.646914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.646914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:00.646914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:00.646914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.646914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:00.646914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.646914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:00.652526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:00.653011 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:00.653142 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:00.653272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:00.653637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:00.653837 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.654258 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:00.654258 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.654258 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:00.654258 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:00.654258 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.654258 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:00.654258 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.654258 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:00.654930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:00.655220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.655220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:00.655220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.655220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:00.655220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:00.655220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.655220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:00.655220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.655220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.655701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:00.655919 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:00.656190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.656190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:00.656190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.656190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.656710 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:00.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:00.657283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.657283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:00.657283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.657283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:00.663858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:00.664176 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:00.664302 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:00.664819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:00.664819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:00.664977 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:00.665109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.665553 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:00.665553 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.665553 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:00.665553 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:00.665553 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.665553 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:00.665553 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.665553 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.666275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.666504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:00.666779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.666779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:00.667134 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:00.667693 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:00.668057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:00.668057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:00.668057 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:05.392094 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:05.392520 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:05.392716 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:05.393556 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:05.424784 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:05.425022 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.425645 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:05.425645 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.425645 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:05.425645 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:05.425645 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.425645 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:05.425645 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.425645 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.426779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.427035 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:05.427245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.427245 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:05.427972 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:05.428255 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.428597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:05.428597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.428597 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:05.638925 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:05.639179 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:05.639279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:05.639710 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:05.639845 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.640236 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:05.640236 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.640236 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:05.640236 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:05.640236 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.640236 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:05.640236 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.640236 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.640852 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.641116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:05.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.641331 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:05.641544 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:05.641869 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.642220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:05.642220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.642220 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:05.643165 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:05.643320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:05.643407 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:05.643492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:05.643736 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:05.643865 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.644182 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:05.644182 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.644182 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:05.644182 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:05.644182 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.644182 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:05.644182 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.644182 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:05.644667 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:05.644970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.644970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:05.644970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.644970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:05.644970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:05.644970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.644970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:05.644970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.644970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.645241 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:05.645595 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:05.645858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.645858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:05.645858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.645858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:05.645936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.646246 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:05.646551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.646551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:05.646551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:05.646551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.646551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:05.646551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.646551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:05.652970 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:05.653159 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:05.653277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:05.653415 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:05.653895 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:05.654121 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.654719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:05.654719 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.654719 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:05.654719 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:05.654719 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.654719 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:05.654719 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.654719 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:05.655265 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:05.655717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.655717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:05.655717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.655717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:05.655717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:05.655717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.655717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:05.655717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.655717 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.655905 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:05.656119 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:05.656373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.656373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:05.656373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.656373 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.656668 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:05.656971 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:05.657279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.657279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:05.657279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.657279 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:05.664188 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:05.664530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:05.664647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:05.664841 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:05.665371 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:05.665604 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:05.665762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.666317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:05.666317 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.666317 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:05.666317 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:05.666317 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.666317 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:05.666317 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.666317 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.667003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.667252 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:05.667470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.667470 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:05.667780 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:05.668053 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:05.668379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:05.668379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:05.668379 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:10.392619 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:10.393137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:10.393137 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:10.393665 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:10.420130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:10.420342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.420705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:10.420705 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.420705 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:10.420705 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:10.420705 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.420705 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:10.420705 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.420705 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.421402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.421917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:10.422377 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:10.422897 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.423573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:10.423573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:10.423573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.423573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:10.423573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.423573 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:10.639435 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:10.639996 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:10.640171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:10.640957 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:10.641115 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.641634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:10.641634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.641634 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:10.641634 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:10.641634 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.641634 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:10.641634 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.641634 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.642676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.642966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:10.643191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.643191 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:10.643835 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:10.644162 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.644471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:10.644471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.644471 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:10.644471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:10.644471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:10.644471 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:10.644603 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:10.644978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:10.645139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.645574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:10.645574 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.645574 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:10.645574 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:10.645574 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.645574 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:10.645574 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.645574 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:10.646457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:10.646856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.646856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:10.646856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.646856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:10.646856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:10.646856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.646856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:10.646856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.646856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.647404 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:10.647640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:10.647927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.647927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:10.647927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.647927 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:10.648523 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.648832 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:10.649128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.649128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:10.649128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:10.649128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.649128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:10.649128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.649128 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:10.653312 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:10.653637 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:10.653960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:10.654287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:10.654485 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.654943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:10.654943 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.654943 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:10.654943 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:10.654943 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.654943 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:10.654943 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.654943 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:10.655871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:10.656135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.656135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:10.656135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.656135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:10.656135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:10.656135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.656135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:10.656135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.656135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.656871 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:10.657085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:10.657401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.657401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:10.657401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.657401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.657898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:10.658196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:10.658656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.658656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:10.658656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.658656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:10.664609 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:10.664914 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:10.665009 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:10.665144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:10.665431 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:10.665616 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:10.665731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.666143 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:10.666143 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.666143 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:10.666143 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:10.666143 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.666143 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:10.666143 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.666143 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.666803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.667054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:10.667273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.667273 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:10.667589 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:10.667898 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:10.668200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:10.668200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:10.668200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:15.393107 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:15.393706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:15.393706 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:15.394330 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:15.425368 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:15.425368 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:15.426551 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:15.426999 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:15.427303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:15.427601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:15.428003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.428003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:15.428003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.428003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:15.428003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:15.428003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.428003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:15.428003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.428003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:15.639996 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:15.640192 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:15.640291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:15.640746 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:15.640931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:15.641381 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.641381 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:15.641381 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:15.641381 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.641381 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:15.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.641381 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.642037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.642240 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:15.642432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.642432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:15.642728 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:15.643036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.643349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:15.643349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.643349 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:15.644373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:15.644554 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:15.644652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:15.644743 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:15.644991 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:15.645129 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.645462 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:15.645462 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.645462 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:15.645462 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:15.645462 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.645462 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:15.645462 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.645462 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:15.646005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:15.646313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.646313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:15.646313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.646313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:15.646313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:15.646313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.646313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:15.646313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.646313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.646705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:15.647169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:15.647452 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.647685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.647976 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:15.653581 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:15.653850 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:15.653949 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:15.654058 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:15.654276 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:15.654460 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.654773 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:15.654773 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.654773 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:15.654773 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:15.654773 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.654773 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:15.654773 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.654773 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:15.655226 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:15.655451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.655451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:15.655451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.655451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:15.655451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:15.655451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.655451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:15.655451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.655451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.655882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:15.656402 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:15.656674 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:15.656895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.657168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:15.664929 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:15.665147 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:15.665231 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:15.665357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:15.665577 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:15.665757 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:15.665842 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.666164 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:15.666164 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.666164 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:15.666164 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:15.666164 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.666164 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:15.666164 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.666164 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.666697 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.666907 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:15.667105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.667105 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:15.667385 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:15.667963 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:15.668319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:15.668319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:15.668319 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:20.393723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:20.394242 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:20.394279 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:20.395146 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:20.424254 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:20.424511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.425140 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:20.425140 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.425140 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:20.425140 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:20.425140 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.425140 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:20.425140 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.425140 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.425809 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.426059 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:20.426272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.426272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:20.426838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:20.427127 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.427475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:20.427475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.427475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:20.640448 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:20.640774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:20.640879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:20.641614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:20.641952 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.642641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:20.642641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.642641 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:20.642641 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:20.642641 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.642641 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:20.642641 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.642641 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.643339 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.643818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:20.644264 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:20.644723 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.645206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:20.645206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:20.645206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.645206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:20.645206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.645206 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:20.645206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:20.645206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:20.645206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:20.645206 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:20.645539 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:20.645783 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.646302 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:20.646302 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.646302 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:20.646302 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:20.646302 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.646302 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:20.646302 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.646302 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:20.646909 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:20.647356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.647356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:20.647356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.647356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:20.647356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:20.647356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.647356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:20.647356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.647356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.647596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:20.647928 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:20.648275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.648612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.649157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:20.653981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:20.654228 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:20.654336 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:20.654432 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:20.654994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:20.655261 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.655806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:20.655806 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.655806 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:20.655806 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:20.655806 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.655806 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:20.655806 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.655806 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:20.656396 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:20.656818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.656818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:20.656818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.656818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:20.656818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:20.656818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.656818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:20.656818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.656818 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.657084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:20.657366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:20.657759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:20.658116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.658504 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:20.665396 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:20.665734 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:20.665892 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:20.666133 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:20.666649 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:20.667023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:20.667157 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.667799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:20.667799 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.667799 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:20.667799 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:20.667799 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.667799 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:20.667799 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.667799 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.668719 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.669084 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:20.669425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.669425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:20.669893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:20.670207 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:20.670705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:20.670705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:20.670705 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:25.394154 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:25.394819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:25.394819 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:25.395323 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:25.423015 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:25.423281 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.423850 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:25.423850 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.423850 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:25.423850 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:25.423850 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.423850 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:25.423850 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.423850 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.424571 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.424939 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:25.425274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:25.425681 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.426212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:25.426212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:25.426212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.426212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:25.426212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.426212 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:25.640894 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:25.641191 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:25.641271 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:25.641911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:25.642135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.642587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:25.642587 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.642587 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:25.642587 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:25.642587 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.642587 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:25.642587 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.642587 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.643293 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.643650 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:25.643961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.643961 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:25.644021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:25.644464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.644969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:25.644969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.644969 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:25.644969 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:25.645270 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:25.645270 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:25.645357 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:25.645723 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:25.645947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.646474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:25.646474 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.646474 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:25.646474 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:25.646474 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.646474 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:25.646474 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.646474 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:25.646992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:25.647426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.647426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:25.647426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.647426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:25.647426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:25.647426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.647426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:25.647426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.647426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.647678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:25.647992 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:25.648322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.648682 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.649145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:25.654291 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:25.654504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:25.654591 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:25.654713 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:25.655069 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:25.655267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.655676 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:25.655676 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.655676 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:25.655676 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:25.655676 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.655676 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:25.655676 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.655676 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:25.656229 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:25.656590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.656590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:25.656590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.656590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:25.656590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:25.656590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.656590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:25.656590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.656590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.656819 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:25.657113 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:25.657441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.657441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:25.657441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.657441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.657703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:25.658500 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:25.658950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.658950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:25.658950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.658950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:25.665751 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:25.665942 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:25.666051 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:25.666220 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:25.666490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:25.666677 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:25.666779 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.667174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:25.667174 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.667174 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:25.667174 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:25.667174 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.667174 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:25.667174 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.667174 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.667876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:25.668521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.668521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:25.668580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:25.668968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:25.669442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:25.669442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:25.669442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:30.394652 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:30.395295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:30.395295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:30.395849 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:30.419427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:30.419653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.420169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:30.420169 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.420169 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:30.420169 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:30.420169 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.420169 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:30.420169 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.420169 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.420887 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.421125 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:30.421325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.421325 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:30.421642 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:30.422036 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.422326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:30.422326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.422326 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:30.641277 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:30.641490 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:30.641617 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:30.642026 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:30.642175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.642581 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:30.642581 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.642581 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:30.642581 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:30.642581 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.642581 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:30.642581 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.642581 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.643203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.643464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:30.643752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.643752 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:30.643933 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:30.644290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.644610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:30.644610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.644610 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:30.645249 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:30.645344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:30.645442 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:30.645529 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:30.645774 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:30.645904 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.646243 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:30.646243 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.646243 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:30.646243 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:30.646243 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.646243 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:30.646243 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.646243 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:30.646750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:30.647001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.647001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:30.647001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.647001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:30.647001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:30.647001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.647001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:30.647001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.647001 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.647342 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:30.647687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:30.647895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.647895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:30.647895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.647895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:30.648248 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.648683 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:30.648983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.648983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:30.648983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:30.648983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.648983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:30.648983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.648983 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:30.654572 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:30.654752 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:30.654863 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:30.654979 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:30.655209 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:30.655364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.655706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:30.655706 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.655706 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:30.655706 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:30.655706 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.655706 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:30.655706 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.655706 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:30.656194 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:30.656432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.656432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:30.656432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.656432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:30.656432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:30.656432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.656432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:30.656432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.656432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.656764 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:30.656979 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:30.657225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.657225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:30.657225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.657225 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.657596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:30.657991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:30.658318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.658318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:30.658318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.658318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:30.666000 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:30.666171 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:30.666272 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:30.666379 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:30.666612 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:30.666794 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:30.666888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.667251 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:30.667251 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.667251 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:30.667251 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:30.667251 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.667251 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:30.667251 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.667251 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.667814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.668006 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:30.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.668223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:30.668455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:30.668760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:30.669043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:30.669043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:30.669043 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:35.395082 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:35.395526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:35.395526 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:35.395983 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:35.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:35.421702 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.422318 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:35.422318 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.422318 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:35.422318 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:35.422318 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.422318 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:35.422318 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.422318 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.423050 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.423372 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:35.423574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.423574 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:35.423826 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:35.424435 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.424726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:35.424726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.424726 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:35.641721 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:35.642028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:35.642028 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:35.642546 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:35.642811 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.643165 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:35.643165 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.643165 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:35.643165 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:35.643165 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.643165 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:35.643165 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.643165 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.643873 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.644291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:35.644594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.644594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:35.644787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:35.645420 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.645911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:35.645911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.645911 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:35.645911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:35.645911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:35.645911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:35.645911 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:35.646156 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:35.646190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:35.646558 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.646558 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:35.646558 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:35.646558 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.646558 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:35.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.646558 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:35.647093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:35.647706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.648238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.648685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:35.649060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.649384 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:35.649957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.649957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:35.649957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:35.649957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.649957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:35.649957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.649957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:35.654818 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:35.655030 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:35.655130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:35.655259 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:35.655559 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:35.655760 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.656131 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:35.656131 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.656131 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:35.656131 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:35.656131 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.656131 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:35.656131 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.656131 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:35.656740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:35.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:35.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:35.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:35.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:35.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.656997 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.657533 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:35.657748 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:35.658052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.658052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:35.658052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.658052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:35.658608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:35.658958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.658958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:35.658958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.658958 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:35.666251 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:35.666433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:35.666521 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:35.666666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:35.666916 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:35.667105 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:35.667190 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.667565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:35.667565 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.667565 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:35.667565 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:35.667565 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.667565 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:35.667565 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.667565 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.668153 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.668388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:35.668612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.668612 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:35.668864 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:35.669135 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:35.669449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:35.669449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:35.669449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:40.395446 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:40.395956 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:40.396077 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:40.396758 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:40.421496 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:40.421743 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:40.422285 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.422285 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:40.422285 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:40.422285 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.422285 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:40.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.422285 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.422993 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.423356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:40.423807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:40.424313 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.424931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:40.424931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:40.424931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.424931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:40.424931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.424931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:40.642173 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:40.642538 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:40.642633 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:40.643196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:40.643406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.643881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:40.643881 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.643881 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:40.643881 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:40.643881 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.643881 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:40.643881 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.643881 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.644611 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.644998 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:40.645359 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:40.645820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.646346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:40.646346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:40.646346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.646346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:40.646346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.646346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:40.646346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:40.646346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:40.646346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:40.646346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:40.646657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:40.646657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.647011 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:40.647011 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.647011 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:40.647011 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:40.647011 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.647011 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:40.647011 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.647011 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:40.647526 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:40.647987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.647987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:40.647987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.647987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:40.647987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:40.647987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.647987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:40.647987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.647987 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.648249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:40.648608 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:40.649005 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.649401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:40.655182 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:40.655444 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:40.655544 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:40.655666 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:40.656063 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:40.656322 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.656793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:40.656793 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.656793 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:40.656793 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:40.656793 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.656793 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:40.656793 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.656793 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:40.657291 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:40.657740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.657740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:40.657740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.657740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:40.657740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:40.657740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.657740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:40.657740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.657740 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.658095 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:40.658443 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:40.658759 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:40.659156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.659757 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:40.666645 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:40.666944 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:40.667045 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:40.667184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:40.667569 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:40.667790 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:40.667908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.668346 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:40.668346 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.668346 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:40.668346 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:40.668346 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.668346 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:40.668346 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.668346 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.669079 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.669335 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:40.669563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.669563 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:40.669731 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:40.670184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:40.670709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:40.670709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:40.670709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:45.395978 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:45.396427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:45.396427 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:45.396879 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:45.420023 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:45.420230 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.420825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:45.420825 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.420825 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:45.420825 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:45.420825 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.420825 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:45.420825 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.420825 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.421508 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.421753 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:45.421950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.421950 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:45.422189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:45.422560 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.422903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:45.422903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.422903 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:45.642700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:45.642998 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:45.643132 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:45.643875 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:45.644100 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:45.644749 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.644749 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:45.644749 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:45.644749 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.644749 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:45.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.644749 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.645493 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.645790 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:45.646060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.646060 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:45.646324 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:45.646619 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.646970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:45.646970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.646970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:45.646970 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:45.646970 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:45.646970 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:45.646970 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:45.647259 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:45.647467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.647970 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:45.647970 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.647970 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:45.647970 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:45.647970 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.647970 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:45.647970 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.647970 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:45.648501 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:45.648770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.648770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:45.648770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.648770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:45.648770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:45.648770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.648770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:45.648770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.648770 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.649296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:45.649519 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:45.649745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.649745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:45.649745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.649745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:45.650296 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.650567 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:45.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:45.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:45.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:45.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:45.655574 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:45.655886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:45.655968 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:45.656072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:45.656437 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:45.656680 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.657189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:45.657189 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.657189 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:45.657189 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:45.657189 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.657189 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:45.657189 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.657189 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:45.657794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:45.658150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.658150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:45.658150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.658150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:45.658150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:45.658150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.658150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:45.658150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.658150 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.658543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:45.659108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:45.659685 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:45.659957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.660238 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:45.667019 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:45.667333 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:45.667448 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:45.667578 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:45.667941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:45.668164 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:45.668275 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.668856 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:45.668856 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.668856 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:45.668856 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:45.668856 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.668856 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:45.668856 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.668856 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.669756 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.669975 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:45.670168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.670168 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:45.670480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:45.670794 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:45.671075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:45.671075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:45.671075 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:50.396389 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:50.396904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:50.396904 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:50.397287 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:50.421324 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:50.421634 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.422317 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:50.422317 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.422317 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:50.422317 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:50.422317 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.422317 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:50.422317 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.422317 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.423003 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.423262 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:50.423524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.423524 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:50.423687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:50.424028 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.424354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:50.424354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.424354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:50.643233 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:50.643595 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:50.643767 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:50.644534 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:50.644801 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.645434 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:50.645434 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.645434 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:50.645434 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:50.645434 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.645434 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:50.645434 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.645434 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.646686 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.647044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:50.647344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.647344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:50.647949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:50.648362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:50.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:50.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:50.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:50.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:50.648915 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:50.649196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:50.649407 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.649966 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:50.649966 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.649966 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:50.649966 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:50.649966 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.649966 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:50.649966 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.649966 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:50.650862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:50.651306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.651306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:50.651306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.651306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:50.651306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:50.651306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.651306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:50.651306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.651306 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:50.652541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:50.653249 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.653798 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.654299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:50.656031 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:50.656296 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:50.656469 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:50.656593 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:50.656986 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:50.657215 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.657910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:50.657910 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.657910 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:50.657910 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:50.657910 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.657910 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:50.657910 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.657910 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:50.658787 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:50.659198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.659198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:50.659198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.659198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:50.659198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:50.659198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.659198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:50.659198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.659198 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.659547 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:50.659914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:50.660250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.660250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:50.660250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.660250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.660367 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:50.660874 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:50.661355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.661355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:50.661355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.661355 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:50.667515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:50.667889 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:50.668007 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:50.668205 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:50.668732 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:50.669003 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:50.669136 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.669745 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:50.669745 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.669745 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:50.669745 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:50.669745 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.669745 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:50.669745 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.669745 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.670841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.671414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:50.671831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.671831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:50.671915 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:50.672400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:50.672840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:50.672840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:50.672840 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:55.396813 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:55.397320 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:55.397393 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:55.398187 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:55.402675 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG dbcounter [-] [92285] Writing DB stats neutron:SELECT=240 {{(pid=92285) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 27 16:26:55.426910 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:55.427145 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.427771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:55.427771 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.427771 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:55.427771 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:55.427771 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.427771 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:55.427771 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.427771 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.428473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.428908 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:55.429310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.429310 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:55.429401 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:55.429891 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.430411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:55.430411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.430411 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:55.643773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:55.644180 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:55.644352 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:55.645130 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:55.645320 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.645849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:55.645849 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.645849 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:55.645849 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:55.645849 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.645849 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:55.645849 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.645849 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.646530 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.646804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:55.647072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.647072 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:55.647286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:55.647591 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.647932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:55.647932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.647932 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:55.648478 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:55.648715 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:55.648798 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:55.648886 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:26:55.649186 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:55.649356 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.649807 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:55.649807 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.649807 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:55.649807 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:55.649807 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.649807 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:55.649807 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.649807 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:55.650294 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:55.650554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.650554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:55.650554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.650554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:55.650554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:55.650554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.650554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:55.650554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.650554 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.650841 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:55.651067 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:55.651301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.651301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:55.651301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.651301 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:55.651492 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.651803 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:55.652099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.652099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:55.652099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:55.652099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.652099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:55.652099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.652099 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:55.656370 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:55.656614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:55.656726 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:55.656818 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:55.657127 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:55.657311 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.657738 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:55.657738 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.657738 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:55.657738 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:55.657738 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.657738 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:55.657738 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.657738 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:26:55.658250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:26:55.658489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.658489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:26:55.658489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.658489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:26:55.658489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:26:55.658489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.658489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:26:55.658489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.658489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.658861 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:55.659055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:55.659278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.659278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:55.659278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.659278 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.659585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:55.659888 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:55.660193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.660193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:55.660193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.660193 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:26:55.667943 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:26:55.668225 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:26:55.668310 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:26:55.668476 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:26:55.668981 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:55.669265 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:26:55.669406 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.670014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:26:55.670014 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.670014 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:26:55.670014 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:26:55.670014 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.670014 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:26:55.670014 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.670014 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.670806 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.671152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:26:55.671466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.671466 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:26:55.671793 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:26:55.672109 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:26:55.672455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:26:55.672455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:26:55.672455 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:00.397284 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:00.397856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:00.397856 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:00.398515 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:00.430223 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:00.430223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.430762 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:00.430762 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.430762 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:00.430762 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:00.430762 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.430762 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:00.430762 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.430762 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.431584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.431849 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:00.432087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.432087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:00.432456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:00.432858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.433167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:00.433167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.433167 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:00.644196 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:00.644564 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:00.644728 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:00.645158 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:00.645303 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.645750 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:00.645750 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.645750 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:00.645750 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:00.645750 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.645750 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:00.645750 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.645750 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.646392 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.646687 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:00.646949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.646949 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:00.647138 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:00.647474 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.647797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:00.647797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.647797 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:00.648802 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:00.649057 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:00.649136 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:00.649230 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:27:00.649458 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:00.649587 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:00.649936 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.649936 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:00.649936 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:00.649936 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.649936 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:00.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.649936 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:00.650578 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:00.650868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.650868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:00.650868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.650868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:00.650868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:00.650868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.650868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:00.650868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.650868 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.651242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:00.651709 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:00.652122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.652362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.652694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:00.656714 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:00.656989 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:00.657081 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:00.657184 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:00.657773 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:00.657924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.658316 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:00.658316 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.658316 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:00.658316 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:00.658316 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.658316 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:00.658316 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.658316 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:00.658931 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:00.659177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.659177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:00.659177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.659177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:00.659177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:00.659177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.659177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:00.659177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.659177 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.659656 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:00.660040 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:00.660476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.660476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:00.660476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.660476 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.660786 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:00.661055 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:00.661344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.661344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:00.661344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.661344 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:00.668297 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:00.668646 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:00.668816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:00.669017 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:00.669352 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:00.669565 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:00.669703 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.670202 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:00.670202 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.670202 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:00.670202 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:00.670202 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.670202 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:00.670202 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.670202 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.671283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.671548 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:00.671831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.671831 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:00.672516 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:00.672825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:00.673158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:00.673158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:00.673158 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:05.397816 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:05.398322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:05.398322 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:05.398700 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:05.424585 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:05.424825 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.425333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:05.425333 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.425333 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:05.425333 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:05.425333 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.425333 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:05.425333 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.425333 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.426044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.426309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:05.426531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.426531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:05.426799 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:05.427399 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.428007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:05.428007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.428007 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:05.644614 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:05.645025 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:05.645150 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:05.645680 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:05.645876 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.646333 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:05.646333 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.646333 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:05.646333 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:05.646333 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.646333 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:05.646333 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.646333 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.647196 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.647454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:05.647746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.647746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:05.648097 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:05.648436 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.648813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:05.648813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.648813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:05.649110 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:05.649314 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:05.649406 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:05.649504 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:27:05.649817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:05.649967 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.650360 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:05.650360 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.650360 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:05.650360 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:05.650360 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.650360 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:05.650360 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.650360 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:05.650996 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:05.651304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.651304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:05.651304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.651304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:05.651304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:05.651304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.651304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:05.651304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.651304 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.651654 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:05.651906 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:05.652142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.652142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:05.652142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.652142 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:05.652456 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.652771 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:05.653073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.653073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:05.653073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:05.653073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.653073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:05.653073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.653073 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:05.657035 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:05.657259 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:05.657351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:05.657456 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:05.657735 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:05.657895 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.658309 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:05.658309 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.658309 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:05.658309 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:05.658309 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.658309 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:05.658309 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.658309 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:05.658879 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:05.659152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.659152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:05.659152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.659152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:05.659152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:05.659152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.659152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:05.659152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.659152 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.659454 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:05.659688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:05.659943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.659943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:05.659943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.659943 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.660181 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:05.660507 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:05.660851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.660851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:05.660851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.660851 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:05.668552 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:05.668750 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:05.668833 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:05.668960 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:05.669180 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:05.669346 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:05.669429 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.669813 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:05.669813 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.669813 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:05.669813 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:05.669813 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.669813 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:05.669813 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.669813 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.670354 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.670598 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:05.670893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.670893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:05.671045 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:05.671332 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:05.671679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:05.671679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:05.671679 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:10.398235 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:10.399072 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:10.399210 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:10.399793 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:10.423093 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:10.423231 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.423596 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:10.423596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.423596 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:10.423596 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:10.423596 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.423596 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:10.423596 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.423596 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.024 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.424203 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.424447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:10.424658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.424658 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:10.424886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:10.425159 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.425469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:10.425469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.425469 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:10.645253 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:10.645572 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:10.645791 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:10.646351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:10.646521 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.647061 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:10.647061 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.647061 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:10.647061 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:10.647061 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.647061 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:10.647061 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.647061 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.647947 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.648366 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:10.648678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.648678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:10.648957 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:10.649388 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.649862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:10.649862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.649862 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:10.649862 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:10.649862 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:10.649862 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:10.649862 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:27:10.650153 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:10.650292 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.650640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:10.650640 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.650640 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:10.650640 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:10.650640 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.650640 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:10.650640 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.650640 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:10.651480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:10.651882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.651882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:10.651882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.651882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:10.651882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:10.651882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.651882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:10.651882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.651882 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.652189 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:10.652699 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:10.653032 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.653362 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.653802 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:10.657283 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:10.657451 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:10.657535 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:10.657613 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:10.657905 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:10.658069 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.658393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:10.658393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.658393 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:10.658393 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:10.658393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.658393 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:10.658393 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.658393 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:10.658900 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:10.659274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.659274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:10.659274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.659274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:10.659274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:10.659274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.659274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:10.659274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.659274 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.659498 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:10.659830 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:10.660141 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:10.660475 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.660929 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:10.668799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:10.668994 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:10.669080 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:10.669247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:10.669492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:10.669683 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:10.669768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.670132 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:10.670132 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.670132 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:10.670132 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:10.670132 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.670132 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:10.670132 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.670132 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.670694 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.671016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:10.671299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.671299 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:10.671426 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:10.672052 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:10.672766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:10.672766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:10.672766 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:15.399212 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:15.399814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:15.399814 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:15.400366 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:15.425851 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:15.426066 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.426742 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:15.426742 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.426742 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:15.426742 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:15.426742 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.426742 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:15.426742 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.426742 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.427614 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.427959 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:15.428283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.428283 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:15.428754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:15.429085 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.429397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:15.429397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.429397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:15.645606 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:15.645941 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:15.646074 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:15.646724 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:15.646941 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.647432 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:15.647432 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.647432 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:15.647432 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:15.647432 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.647432 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:15.647432 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.647432 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.648449 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.648815 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:15.649122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.649122 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:15.649788 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:15.650209 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.650657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:15.650657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.650657 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:15.650657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:15.650657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:15.650657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:15.650657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:27:15.650657 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:15.650881 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.651139 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:15.651139 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.651139 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:15.651139 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:15.651139 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.651139 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:15.651139 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.651139 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:15.651924 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:15.652272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.652272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:15.652272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.652272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:15.652272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:15.652272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.652272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:15.652272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.652272 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.652736 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:15.653021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:15.653286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.653286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:15.653286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.653286 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:15.653782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.654174 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:15.654541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.654541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:15.654541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:15.654541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.654541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:15.654541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.654541 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:15.657563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:15.657725 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:15.657830 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:15.657973 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:15.658313 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:15.658511 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.658925 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:15.658925 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.658925 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:15.658925 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:15.658925 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.658925 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:15.658925 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.658925 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:15.659678 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:15.659940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.659940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:15.659940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.659940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:15.659940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:15.659940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.659940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:15.659940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.659940 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.660543 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:15.660755 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:15.660994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.660994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:15.660994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.660994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.661600 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:15.661965 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:15.662244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.662244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:15.662244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.662244 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:15.669052 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:15.669190 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:15.669301 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:15.669450 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:15.669917 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:15.670109 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:15.670200 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.670700 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:15.670700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.670700 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:15.670700 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:15.670700 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.670700 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:15.670700 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.670700 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.671505 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.671782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:15.671991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.671991 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:15.672451 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:15.672768 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:15.673089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:15.673089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:15.673089 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:20.399681 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:20.400189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:20.400189 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:20.400644 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:20.425474 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:20.425688 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.426223 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:20.426223 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.426223 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:20.426223 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:20.426223 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.426223 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:20.426223 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.426223 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.427108 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.427473 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:20.427816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.427816 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:20.428400 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:20.429016 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.429754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:20.429754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.429754 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:20.646115 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:20.646365 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:20.646425 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:20.647012 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:20.647233 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.647782 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:20.647782 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.647782 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:20.647782 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:20.647782 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.647782 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:20.647782 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.647782 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.648513 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.648776 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:20.649019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.649019 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:20.649414 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:20.650247 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.650701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:20.650701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.650701 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:20.650701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:20.650701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:20.650701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:20.650701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:27:20.650701 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:20.650853 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.651270 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:20.651270 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.651270 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:20.651270 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:20.651270 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.651270 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:20.651270 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.651270 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:20.651838 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:20.652104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.652104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:20.652104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.652104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:20.652104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:20.652104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.652104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:20.652104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.652104 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.652459 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:20.652715 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:20.652946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.652946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:20.652946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.652946 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:20.653093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.653446 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:20.653744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.653744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:20.653744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:20.653744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.653744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:20.653744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.653744 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:20.657807 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:20.657962 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:20.658048 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:20.658144 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:20.658433 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:20.658631 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.659014 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:20.659014 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.659014 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:20.659014 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:20.659014 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.659014 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:20.659014 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.659014 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:20.659499 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:20.659789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.659789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:20.659789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.659789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:20.659789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:20.659789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.659789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:20.659789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.659789 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.660116 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:20.660330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:20.660601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.660601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:20.660601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.660601 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.660814 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:20.661184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:20.661480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.661480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:20.661480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.661480 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:20.669308 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:20.669448 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:20.669530 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:20.669668 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:20.669964 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:20.670149 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:20.670242 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.670635 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:20.670635 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.670635 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:20.670635 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:20.670635 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.670635 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:20.670635 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.670635 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.671232 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.671447 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:20.671695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.671695 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:20.671886 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:20.672227 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:20.672515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:20.672515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:20.672515 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:25.399972 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:25.400430 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:25.400590 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:25.401344 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:25.425966 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:25.426170 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.426592 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:25.426592 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.426592 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:25.426592 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:25.426592 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.426592 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:25.426592 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.426592 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.427237 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.427457 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:25.427746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.427746 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:25.427968 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:25.428266 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.428570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:25.428570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.428570 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:25.646455 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:25.646837 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:25.646990 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:25.647492 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:25.647651 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.648054 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:25.648054 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.648054 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:25.648054 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:25.648054 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.648054 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:25.648054 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.648054 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.648708 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.649123 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:25.649584 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:25.650092 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.650641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:25.650641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:25.650641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.650641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:25.650641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.650641 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:25.650641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:25.650641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:25.650641 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:25.650937 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:27:25.650937 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:25.651093 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.651393 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:25.651393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.651393 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:25.651393 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:25.651393 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.651393 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:25.651393 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.651393 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:25.651910 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:25.652425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.652425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:25.652425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.652425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:25.652425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:25.652425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.652425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:25.652425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.652425 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.652713 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:25.653082 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:25.653489 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.653893 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.654395 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:25.658053 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:25.658262 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:25.658351 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:25.658453 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:25.658735 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:25.658917 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.659314 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:25.659314 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.659314 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:25.659314 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:25.659314 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.659314 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:25.659314 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.659314 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:25.660015 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:25.660442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.660442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:25.660442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.660442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:25.660442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:25.660442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.660442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:25.660442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.660442 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.660727 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:25.661021 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:25.661397 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:25.661820 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.662370 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:25.669566 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:25.669817 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:25.669914 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:25.670083 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:25.670362 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:25.670558 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:25.670669 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.671037 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:25.671037 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.671037 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:25.671037 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:25.671037 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.671037 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:25.671037 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.671037 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.671580 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.671930 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:25.672330 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:25.672769 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:25.673340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:25.673340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:25.673340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:25.673340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:25.673340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:25.673340 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:30.400563 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:30.401147 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:30.401147 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:30.401647 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:30.425643 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:30.425914 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.426510 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:30.426510 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.426510 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:30.426510 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:30.426510 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.426510 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:30.426510 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.426510 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.427267 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.427512 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:30.427804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.427804 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:30.427994 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:30.428315 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.428706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:30.428706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.428706 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:30.646883 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:30.647247 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:30.647373 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:30.647874 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-984e4ec9-c3a3-461c-8eb1-0258810f9538, routes=[], if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:30.648020 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.648441 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:30.648441 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.648441 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:30.648441 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:30.648441 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.648441 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:30.648441 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.648441 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.649169 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.649383 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:30.649640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.649640 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:30.650098 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:30.650364 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.650858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:30.650858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.650858 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:30.650858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:30.650858 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:30.650974 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:30.651041 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting fip-distributed flag in NB_Global to False {{(pid=92285) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 27 16:27:30.651325 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:30.651464 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.651828 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:30.651828 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.651828 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:30.651828 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:30.651828 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.651828 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:30.651828 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.651828 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:30.652380 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:30.652653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.652653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:30.652653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.652653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:30.652653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:30.652653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.652653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:30.652653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.652653 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.653102 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:30.653345 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:30.653590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.653590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:30.653590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.653590 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:30.654008 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                                                           ^^^^^^^^
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.654290 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:30.654565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.654565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:30.654565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:30.654565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.654565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:30.654565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.654565 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:30.658334 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:30.658544 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_ha_failover {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:30.658671 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:30.658799 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:30.659062 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:30.659250 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.659594 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:30.659594 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.659594 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:30.659594 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:30.659594 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.659594 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:30.659594 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.659594 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 27 16:27:30.660175 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 27 16:27:30.660488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.660488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 27 16:27:30.660488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.660488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 27 16:27:30.660488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 27 16:27:30.660488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.660488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 27 16:27:30.660488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.660488 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.command 
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.660962 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:30.661328 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:30.661737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.661737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:30.661737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.661737 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.662087 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:30.662664 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:30.663235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.663235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:30.663235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.663235 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics 
May 27 16:27:30.669873 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=92285) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 27 16:27:30.670155 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 27 16:27:30.670295 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=92285) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 27 16:27:30.670481 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=92285) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 27 16:27:30.670730 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:30.670933 np0000001523 neutron-ovn-maintenance-worker[92285]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=92285) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 27 16:27:30.671044 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.671467 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Traceback (most recent call last):
May 27 16:27:30.671467 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.671467 np0000001523 neutron-ovn-maintenance-worker[92285]:     txn.results.put(txn.do_commit())
May 27 16:27:30.671467 np0000001523 neutron-ovn-maintenance-worker[92285]:                     ^^^^^^^^^^^^^^^
May 27 16:27:30.671467 np0000001523 neutron-ovn-maintenance-worker[92285]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.671467 np0000001523 neutron-ovn-maintenance-worker[92285]:     raise RuntimeError(msg)
May 27 16:27:30.671467 np0000001523 neutron-ovn-maintenance-worker[92285]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.671467 np0000001523 neutron-ovn-maintenance-worker[92285]: 
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.672184 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.672531 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 27 16:27:30.672847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.672847 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics [None req-5d732f9c-4318-4860-95d0-3fe294e1df57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics Traceback (most recent call last):
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     work()
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(*args, **kwargs)
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 27 16:27:30.673156 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     next(self.gen)
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     with self.create_transaction(
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     self.result = self.commit()
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise result.ex
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 27 16:27:30.673585 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 27 16:27:30.674137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics     raise RuntimeError(msg)
May 27 16:27:30.674137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 27 16:27:30.674137 np0000001523 neutron-ovn-maintenance-worker[92285]: ERROR futurist.periodics